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COMMON CORE CURRIpULUM OF VOCJ^TIONAL EDUCATION 
WORKSHOP PROCEEDINGS * 



Introduction 



A grant thropgh the Cfllifornia State Department of Education rade 
|.t-;PQSsj^ble to continue the development of additional modules identified 
as appropriate for a Xommon Core Curriculum for Vocational Education. 

Several sources were used to determine topics relevant and common 

to all vocational education programs. The topics Were categorized and 

■ - I 

grouped according to possible module titles. The module waiters were 
CSUf teacher educators from tl\e vocational education services of Agriculture 
Education, Business Education, Home Economics Education, and Industrial 
Education* The writer? selected according to^fheir interest and expertise 
three topics to develop as competency-based instructional modules for 
vocational education,^ 



The first drafts of these modules were completed during the fall 
semester and ready for field review and testing^^lris^pring. Invitations 
to participate in the field testing were sent to all institutions of higher*, 
education offering vocational education programs. It i^ recognized^tliR 
'an instructional modular curriculum allows for a great degree of adaptability 
in ciJse and in application to any local ^ystem of education. However, for 
field testing the instructors were asked to follow similar procedures 
(Appendix Bp ). The main elements of field testiVig consisted of (1) Instructor 
rating of the validity of the content; (2) a record of scores for pre and 
post test completed by student; (3) a studentVs **an Opinion and Satisfaction 
questionnaire, with (4) a student background data sheet-" 



^ See attached list of modules. 

i i i > 



During the latter part of flarch a two*day workshop for module 
writ)^rs and field test Instructors was held to give support arfd feetJback 
to the writers and to clarify soine of the^'questions raised by field test 
instructors, 

J 

The purpose of the workshop was (1) 'to consider in depth research 
methods and needs in vocational education as means of strengthening the 
curriculum for pre and in-service vocational education programs and (2) to 
assess and give advice on the 29 competency-based curriculum modules. 

Instructors from 15 California institutions, were involved in the 
field testing. In addition, all modules -were reviewed by the workshop 
consultant and a specialist in business education, 

For these who attended the vtorkshop, we appreciate your support and 
parttcipatioji. We hope you will find these proceedings useful in your 
vocational education programs, 
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THE APPtlCATION OF THE RESEARCH PROCESS TO 
PROBLEMS IN VOCATIONAL EDUCATION' 

by Dr.. J. Robert Wambrod, Professor 

I begin my remarks by relating incident described to me by a teacher 
in an area vocational school ia Ohio, ,A student, Jim, was enrolled 1n a 
certain vocational course--let's call it Course A—during his junior year. 
He showed little enthusiasm for the course -or for school in general* His 
attendance was poor and? as you wduld expect, his grades including grades 
in the vocational course', were marginal at best* At the end of his junior 
year, Jim elected or was counseled to enroll for a diffevr'ent' course during * 
his senior year—let's- designate this new course as Course iXuring his 
senio^ year* Jim's^attitucfe, motivationv^ and performance were about as 
different from the previous year as possible. He was highly motivated; 
^ he performed exceptionally well both in courses arid on the job. He attended 
school regularly and his grades were the highest he had ever earned- In 

fact, his employer for supervised occupational experience asked schoo| 

, I 

officials at mid-year whether arrangements could be made for Jim to work 
full-time so that Jim could help train other employees. At the end of the 
senior year, Jim was recognized as the most outstanding student in the 
area vocational center. 

I begin with this true episode as a means of encouraging you to begin 
'thinking about research in vocational education, I ask you-^to speculate 
about these questions; Why did Jim change his behavior? What factors can 
explain why Jim performed much better in Course B than in Course A? How 
could we find out why Jim changed his behavior^ We might evefh want to 
consider this question: Is it important to explain why Jim performed diN 
-ferently in the two courses? I will . return to^ this incident near the end 
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of tfie presentation for it illustrates some of the points I want to discuss 
with you about research in vocational education. 

Some persons are somewhat skeptical about the vAlue and rise of research 
in education in general and research in vocational education iff particular. 
Perhaps some of you share that skepticism* On occasion research ^^charac- 
terized as an activity for professors or certain individuals in state 
departments of education. Some persons claim that researchers spend their 
time on insignificant concerns with little attention to practical problems. 
Others, sometimes administrators perhaps* point out that the research process 
is frequently too slow to provide data for decisions thaj^need to be made ' 
today, not three years from now when the project will be^completed. Even 
when the research is completed, there is some evidence that researchers 
tend to write reports that range from difficult to impossible to under- 
stand—at least by persons who are not researchers / We fill our reports with 
terns like ciii square, analysis of variance', multi-stage cluster sampling, 
level of statistical significance; and the like when those who prob^ly 
have the, greatest need for the research may be interested in questions 
such as these; What was the problem investigated? What are the valid 
findings? What are the implications of the firdings for policy and pro- 
gram development or revision? 

I want to share with you some thoughts about the appropriate appli- 
cation of the research process to problems and issOes in vocational 
education. My position is that research is of utmost importance to those 
of us in vocational education. It is essential that we understand the 
process and use it appropriately. 



What Is Resear ch? ^ ^ 

What is research? We can find a very scholarly written definition in 
any Introductory textbook on research methodology. Instead of reciting a 
definition of research, I propose that we look at the process from a slightly 
different point of view. A goal of research tha^ is of major concern to us 
is that research needs to betllrectly connected with practice, that is, the 
new knowledge and new products developed through t^e research process need 
to make direct contiributions to policy and practice In vocational education 
pr(>3rams. We conduct research that contributes to the development of new 
practices, products, programs, and pQlicles, We Install or diopt new pro- 

^ grams and policies theri conduct research to see if the hoped-for outcomes . 
are actually achieved- In effect, Wc use the research process to provide 
data for decision making; The research process can be used to provide 
infomation that can be used in formulating and revising policy, in develop- 
inq programs and instructional products* and In evaluating whether or not . 

.the outcomes expected are actually accomplished. 

The definition of research thiit 'I like Is one given by the late 

Professor H. M. Hamlin of the University of Illinois: 

> 

Research is an unusually stubborn and persisting effort to 
think straight, which involves the gathering and the intelligent ^ 
:;ysa of relevant data. ( American Vocational Journal , September 
1966) 

Note that research is a process,^ It is a process that begins with a 
feeling of perplexity, with a hunch that something could be changed, or 
with a curipsity about an Idea or an event that is not satisfied. The 
process involves a clear and precise definition of the problem to be 
investigated, a search of'what is already known about possible solutions or 
answers, and the formulation and testing of alternative solutions. Here 
is where we collect.and intelligently use relevant data. A part of the 



process that we slight sometimes 1s the reporting of the research. It is 

Important that we rertember ^/lat the research process Is not complete until 

both the process and the outcomes of the process are made public. 

You know the research process as the" method of sc1ence--the scientific 

method. Some persons perceive the scientific method as a dull* laborious; 

and Infrequently used process.. On the contrary* all of us use t}\e process 

many times each day as we formulate and test alternatives and as we make 

decisions, John Dewey described the systematic inquiry process as 

'*^?eflective thinking." I like this desc^riptlon of the method of science. 

People talk about '*the scientific method** e^S if it were 
some'magit computer*-like method of solving all of our thinking 
problems. There is nothing magical about it, but only a slightly 
formal kind of common sense: Wh^t is the question? Shall I seek 
, the answer through experimentation* or reading what different 
' people have discovered? how do rny results stack up after con- 
sidering the negatives as well as the positives? where can I go 
from this answer so as to evolve my own. theory or solution? 
, CRoyal Bank of Canada Monthly Letter* February 1971) 

Selecting Problems for Researc h 

How do we operational ize, then, this "sllgfitly formal kind of dommon 
sense'^'^proach in the investigation of problems in vocational education? 
How do'^^e apply straight-thinking in a "stubborn and persistent*' manner as 
we design, conduct, and report research? 

The beginning point must be the identification and selection of signi- 
ficant and relevant 'problems and issues. Professor Hamlin, who I quoted 
earlier, said that "Ideas and concepts are tTie first requisite for good 
research. Research without important idea^ is. busywork.'* One legitimate 
criticism of research in education* including research in vocational educa- 
tion* is that it does not deal with important *'real-world*' problems. You 
have a unique opportunity to remedy that for you are engrossed in these 
**real-world" problems every day. Perhaps your dilemma is that of choosing 



from an array of problems and Issues those that warrant highest priority 

for study and investigation. 

I suggest that an appropriate strategy for selecting significant 

problems fo research in vocational education is to examine our ^^sacred 

cows," that iSs those practices and policies that we t?ke for granted. 

Are we wtio work in vocational education willing to put our ''sacr^ed cows" 

to the test of systematic inquiry? Occasionally we encounter research on 

what appears to be somewhat outlandish problems. Congressmen and Senators 

are known for c^^J^ing these examples to our attention when they review 

research projects funded by public agencies* My poifit is that we should 

look beyond t^ie title of the research project befor-e making judgment con^ 

cerning the feasibility and significance of the endeavor. Research tl>at 

appears to be outlandish^ or nonsensical may not be as absurd as we think. 

- Let me illustrate. A few years ago an article titled "Talking Eggs'' 

.appeared in a publication of the U. S. Department of Agriculture. The 

article was a report of research designed to investigate whether unhatched 

eggs commynicate with each other* A skeptical and irate reader wrote the 

editor of the publication claiming that there were more appropriate uses 

of public funds than for some scientist to try to find out if eggs communi 

cate with other eggs. The editor asked the scientist to reply to the 

reader's letter. Here is part of what the scientist wrote: 

You're right, sometimes it seems that some of our scientists 
are not using good common sense in some of their research. How- 
ever, modern scientists are trained to look for unusual methods 
and to use sophisticated tools to sj>lve tough problems-. . . 
For example, anyone ysing common horse sense would know that you 
^ can't get anywheVe. incubating eggs from turkeys which have not 
been mated. Yet iti t&53» a^^cientist did just that, and discovered 
that the eggs hatched rfesuTt4ng in fatherlessturkeys. Some people 
may still wonder aliput the value of fatherless turkeys, but they 
have turned out to be of tremendous interest for scient^* 
( Agricultural Situation , September 1970) 



•■ ' ■- . 6 

I use ^th1s example 1n a research methods course I teach. . A couple of 

years ago on the I had used the example in cla5;.., the following hqad- 

line to'a page-one article In the Wall .Street Journal caught iny eye. 

■ " ■ , \ 
The Birds and Bees Can't Explain Births ' 
6f Certai^n Turkeys ^ ■ ' ^ 

Mother hens can have poul-ts and keep their virtue, to*o; , 
Is a virus better than^ sex? Here's a question for you: - Can 
a virgin turkey be a mother? - 

- According to poultry scientists here at Pennsylvania State ^ 
University, the answer is an unequivocal yes\ As a matter of 
fact, they say, virgin births occur so often in turkey circles ^ 
these days, they hardTy raisp eyebrows anymore. 

My point is t lat you are in a position to identify some of the tough 
and significant problems in vocational -education that should be researched." 
Indeed, some of these problems may even be s'omewhat unusual — maybe even" 
outlandish. There are also ordinarjy and routine problems that need ^ 
investigating, i^y argument is that research in )(Ocational education is 
of little value unless the problen being researched i^ worth investigating. 
On occasion* graduate students as(f how they can decide what problem should 
be investigated. They imply that^^h^ method of science involves some ' / 
systematic and objective mechanism whereby significant problems for research 
are identified. Needless to say* they are usually disappointed when I 
suggest that the selection of significant researchable problems is in large 
part the application of good common sense and the use of the be'jt advice 
available from experts. You are some of those experts when it comes to 
identifying relevant problems and issues fpr research in vocational education, 
Types of ResearchBroblems ' ^ ' , 

Remember^ problem identification is only the first step in the process 

V 

of systematic inquiry. Basically* research problems can be categorized into 
three groups based tjpon "the purpose to be accomplished by the research* or 
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stated another way» grouped according to the" intent of the investigator, 
rirst, there are investigations designed primarily to describe groups or 
situations. In vocational education, these are studies designed primarily ^ 
to describe the characteristics, competencies, or performance of persons who 
<ire enrolled in or have completed certain programs. A second category of 
research questions includes concerns such as: What factoids tend to accompaoy 
certain outcomes of educational programs? Why do things happen the wav they 
do? Here the intent of the researcher is to .study the relationship-- 
correlation-^between variables. Vocational education studies of this nature 
frequently involve an investigation of the relatio^nship between outcomes 
of educational programs &nd characteri:itics of studerlts, the nature and 
type of educational ^activities experienced, and the nature of the environ- 
ment or setting in which the educational program is conducted. The third 
category of research qi^estions includes this type of question: What are 
the outcomes*"effects--produced by a certain program? Here the researcher 
wishes to establish a causal reldtionshtp between certain inputs and the 
OLJtcomcs that are observed. 

In research parljfice, studies designed to describe g^^^ps or situations 
are tjsually labeled surveys, or sample surveys; research designed to invesU- 
gate relationships between variables is correlational research; and when 
the intent is to estab! i^h cause-effect rela,tionships, Experimental research 
is called for. " ^x; * 

Research in vocatjonal education is useful .only if properly conducted/ 
and interpreted. I want to call your attentipn to some major and worrisome 
methodological problems usually encountered when conducting research. Thi^s 
afternoon we will take a look at some of these concerns in more detail. 
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Descriptive Research . When the investigator's intent is to describe 
accurately, that Is, when survey research strategies are being used, data 
are usually collected by mall questionnaire or Interview. An Immediate 
concern that must be dealt with is response rate* particularly when mail 
questionnaires are used. How can ari accurate description of a group or 
situation result if data are available from one-half or leSs of those who 
were asked to respond? The researcher must devise ways of getting sufficient 

and non-biased responses if the research is to be valid. In an effort to 

! 

encourage response to a mai.l survey, one school system printed the follow- 
ing on the envelopes in which the questionnaires were mailed. 

Warning: If you throw this in your waste basket unopened, 
a capsule of water inside will break* spilling onto a dehy- 
drated gorilla. He will- then jump out of the envelope and 
hug you to death. 

In descriptive researc', is one thing to get responses, but another 

equally important concern is the accuracy of the data reported. I will' 

illustrate with this conversation between the turtle ^ Ad the bug in the 

Pogo comic strip. > 

Turtle: To eaSe your mind on humans we should take a big fat 
survey. 

Bug: ^ An' then refer the report to a committee? 

Turtle: Right. . .then the committee could issue a prognostic 
preview and Congers could call for a vote. 

Bug: Forestallin* the revolution? 

Turtle; Sure. . .we'd query all types. . .bugs, mongeese, 
antelopes, bandicoots. . .all them. . .and find 
out how many hidden humans there are. 

Bug: We'd ask human beans too? 

Turtle: To be scrupulous fair, yeh. 

Bug CThis is the punch line}: How could you be sure some of 
them wouldn't lie about it? 



We need to be aware that some persons from whom we request data simply 

conclude that what is being asked is none of our business; others want to 

make themselves look good so they tend to answer the way they perceive 

they ar^ expected to answer; andLjhen there are those who, as the bug 

says, '*lie about it.'* Perhaps a solution to these measurement error pro- 

blems is the procedure described in the following discussion between Pogo, 

the possum, and three rats who are employed as pollsters for the local 

survey research agency in the swamp. 

Pogo: So you rats are gonna be pollsters for P.T. 

Bridgeport? 

First rat; Yep. . .altho P, T, don't always play it fair. 

Second rat: Yeh. . .last time we run a poll for him if he 

didn't like our results, he changed the whole thing. 

Third rat; Very embarrassin' . 

Pogo: I should guess so. . .that'd make the prognosti- 

cation wrong? , 

First rat; Well, what was more entarrassin' was it made it 
right. 

Correlational Research . When designing and conducting "correlational 
research, an important concern is the interpretation of the relationships 
that are described. In essence, we must be careful not to interpret 
correlation as causation. 

Let me highlight this concern by describing briefly a study com- 
pleted by a graduate student who was investigating this question: What is 
the relationship between the extent to which high school students participate 
in a vocational students' organization and their capabilities in leader- 
ship, citizenship* and success on the job? The researcher hypothesized that 
the more a student participates in a vocational students' organization* 
the more capable a leader and citizen^ and the more successful on the job 



after completing high schooK What were the findings? As you might wefll 
suspect, the investigator found that the more a student participates in 
activities of a vocational students' organization {FFA, FHA, VICA, and 
others) in high school, the higher he or she is rated in leadership and 
citizenship and the more successful he or she was on thie job. Creditable 
correlational research does not end with that finding and an accompanying 
itrterpretation linking in a direct* if not causal fashion, the degree of 
participation in students' organizations with certain outcomes. The 
proper interpretation of correlational research requires additional 
probing. In this study, the investigator als"6 found that the higher a 
student*s grades in high school, the higher the student was rated on 
leadership, citizenship, and job success. Similar relationships were 
found for variables like socioeconomic status and the number of years of 
vocational education completed. The facts are that there are several 
factors, in addition to extent of participation in a, students' organiza- 
tion, that are related to leadership, citizenship, and job success. But 
note these additional findings. The investigator also demonstrated that 
students participating to the greatest extent in the activities of a 
students' organization were those who earned higher grades, were from 
families with higher socioeconomic status,, and had completed more years of 
vocational education in high schooK Remember, each of these factors 
was also associated with leadership, citizenship, and job success. Note 
that these additional findings substantially qualify, some would say dis- 
qualify, an interpretation that links directly participation in a students* 
organization and students' leadership, citizenship, and job performance 
abtlities^^ . 
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My point is that tl)fe proper interpretation of correlational research 
fs dffficuU. A less serious example Illustrates the pofnt wel4\^ There 
IS a relatively high positive correlation between the salary received by 
the clergy and expenditure for alcoholic beverage^X The bfgher the 
salary peid clergy, the higher the expenditure for alcoholic beverages 
in the coimunity. In the northarn states during the summer months, there 
is a high positive correlation between the monthly number of deaths by 
drowning and the gross monthly sales of ice cream cones* To describe 
the magnitude and direction of these relationships Is one thing, but to - 
claim that deaths by drowning can be eliminated by a law banning ice 
cream parlors makes about as "luch sense as to argue that the salaries of 
clergy will increase if we consume more booze. 

Experimental Research . The findings of experimental research must be 

interpreted with caution also. If we are to draw valid cause-effect ^ 

interpretations, we must take special precautions to rule out plausit^1e 

rival explanations for interpretations that certain factors cause certain 

ouUComes. The difficulty I am calling to your attention is indicated 

fay ths following: 

Dear Abby; I jiave just started to date, and i have a problem ^ 
which Is common to most girls my age. It seems that every time 
that I go out with a boy. If I let him kiss me on the first date 
I never see him again. But then if I don't let him kiss me^ I 
will never see him ffgain either. So how is a girl supposed tp - 
know what to da? Just wo-ndering. 

Dear Just"-She can assume that whether^s'he kisses a boy or not 
on the first date has nothing to do with whether she ever sees 
him again* and go to wor^c on other possible causes for being 
a one-da te-dolly. 

If experimental research is properly designed and conducted, "pther 
possible causes" for the outcomes observed are controlled or discounted; 
hence valid cause-effect interpretations are possible. As producers an5 

7 / 



ERiC 



■J 



I 



~ 12 

consumers of research in vocational education, we must insure thai; 1n 
designing and interpreting experimental and quasi-experimental research 
that precautions are taken to rule out those extraneous or intervening 
factors that invalidate findings stating that the outcomes observed are 
caused by certain factors or programs, 

y Even if the" research is valid—that is, the findings are true— we 
also have to consider the extent to which the findings can fae generalized, 
St^ated more directly, we ask this question: To what groups, to what 
settings and situations, and to what points in time, and to what measure- 
ments can the results be applied? IfTot^her words, can we extrapolate, 
or transport the results to other groups, other situations, other points 
in time, and the like? - v 

I will illustrate with a television and radio commercial with which 
you are probably very familiar, The^early versions of the commercial 
included this statement: ^*Two Excedrin are more effective for the relief 
of pain than twice as many aspirin," When'we hear a statement such as 
this, I suspect that the pain most of us think about is headache. As you 
might expect, at least one manufacturer of aspirin objected to the 
commercial claiming that the information in the commercial was not true; 
In other words, the credibility or validity of the information was 
questioned. The manufacturers of Excedrin defended the commercial claim- 
ing that the information was based tipon an accurate and truthful summary 
of a clinical study done by reputable and expert scientists. They addjed, 
however, that the commercial referred to a different kind of pain— not 
pain suffered by people with splitting headaches but by women who had just 
given birth! So you see* the validity of information — the validity of 
the findings of research--is one concern; but another concern with which ^ 



13 

we must be aware 1s whether or not the Information Is applicable, Is 
generalizable, to groups and environments other than the specific cir- ■ 
cumstances at the time the research was conducted. Incidentally, the 
next time you see or hear that Excedrin commercial* note that the phrase 
'*a different kind of pain" has teen inserted in the commercial. 

Reiteration . Let me reiterate briefly. Basically, research in 
vocational education is designed to answer one or more types of questions. 
Some research has as its major focus this question: What are the facts? 
I label this research descriptive or survey research. Other researchers 
ask the question: What factors explain — what factors are related to-- 
certain outcomes? Research designed to answer these kinds of questions is 
" correlationaJ research. Still others probe^questions like What factors 
cause certain outcomes? That type of question can only be answered 
valxidly through experimental research. Hy point is that interpretation 
of the findings of research must not go beyond what Is Intended by the 
researcher and what is warranted by the design and conduct of the research. 
I suggest that we be more cautious in interpreting research then the 
interpretation indicated by tf^is conversation from a Wizard of Id comic- 
strip episode. 

Wizard (speaking to the King): My studies ^show a direct 

correlation betwi&en increased taxes and crime rates. 

King; What conclusions have you drawn? 

Wizard;^ They can steal it as fast as you can. 

Summary- .^ ^ 

Id suimiiry, what does this assortment of comic-strip wisdom and Dear 
Abby correspondence have to do with tbe application of the research pro- 
cess to problems in, vocational education? I hope it has helped to make 
clear and illustrate three basic points. First, it is important that'we 
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apply the research process 1n planning* Implementing » and evaluating 
programs in vocational education. Remember, research—the scientific 
jrethod"1s a process that requires that we beware of our certainties and 
that we admit that we have doubts. Second, the research process requires 
that we begin with significant and crucial problems that have import both 
for the present and the future of vocational education. And third, the 
process demands that we collect relevant data and that we intelligently 
use and Interpret the data. The research that we conduct and the outcomes 
of that research will be valid only to the extent those of us conducting 
and Interpreting the research make it so. 

Let's return to Jim, the student I referred to at the beginning of 
this presentation. Why did Jim's behavior in school change so dramatical ly 
from his junior to senior year? Was it because Course B was better than 
Course^ A? Maybe the teacher of Course B was more effective; maybe Jim's 
attitude toward school and things in general changed; maybe his parents 
brought about the change. I do not knoW what explanations you propose; 
but if you have fomulated some possible explanations, you are involved 
in the research process. The next step is to collect data to see if your 
hunch--your most plausible explanat1or}--is confirmed. 

I suspect that most of us would prefer an explanation that links the 
new course or the new teacher to Jim^s marked change in behavior. Was ^ 
the new course in which Jim enrolled his senior year the factor accounting 

h 

for his change in behavior? The teacher relating the incident was not 
naive enough to answer "yes-" In fact,\the teacher was perceptive enough 

\ 

to answer "no." The reason for that response was the teacher knew that 
Jim was married between the end of the junior year and the beginning of 
the senior year—a happening prompted* by the way,, by the fact th,at his 



15 

becoming a father was Imminent, This new responsibility* which undoubtedly 
Jim took very seriously, was the factor the teacKer credits as the prirre 
motive for Jim*s change In behavior. In fact* the teacher was very emphati 
and indicated that he was well aware that the same results could have 
been observed had Jim been enrolled in any course In the school. It wasn't 
the course that made the difference; It was Jim who made the difference. 

■ The lesson Is that we look for those "other possible cause^^' when 
we design and use researdi. We must not automatically assume that when we 
get the results we are looking for that the results are always produced 
by the factors we expected. Some of our sessions today and tomorrow are 
designed to Increase our competence as researchers and to help us become 
more discerning consumers of research. That being the case* I am certain 
that you will neither conduct nor interpret research as described by the 
following. 

Test 1 . Remove the front two legs from a grasshopper and plate 
it on a table top. Give the verbal command: Crawl! Result; 
Grasshopper crawled. 

Test 2 . Remove the second set of front legs from the same 
grasshopper and replace it. on the table top. Give the verbal 
command: Crawl! Result: Grasshopper crawled. 

Test 3 . Remove the back legs from the same grasshopper, leaving 
him no legs and replace it on the table top. Give the verbal 
command: Crawl! Result: Grasshopper did not crawl. 

Conclusion : It has been determined beyond all doubt by a series 
of tests using exacting scientific methods that a grasshopper 
with all its legs removed can no longer hear! 
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RESEARCH DESIGN: IMPLICAIIOMS FOR INTERPRETATION 
by or. J* Robert Warmbrod 

During this session I will -build on the general ideas presented 
during the morning session with particular attention to the design of 
research and the inter-relationships between design and interpretation 
of the findings of research. I will attempt to tie this discussion 
with some of the majcr points presented during the morning session. 
Remember, the starting point for research in vocational education is 
significant and important problems for investigation r-**Research without 
important ideas is buSyWork'\ Assuming that we have important problems 
to investigate, now let's coircentrate on designing and conSuicting 
investigations such that valid findings are revealed* 

ril begin with another commercial. "Our group had 20 percent 
fewer (dental) cavitiesl'* All of us have heard thts or a similar ad- 
vertising report of the findings of studies designed to evaluate the 
effectiveness of .toothpaste . When we attempt to interpret this "finding' 
some questions immediately come to mind: With what other group or 
groups was "our group" compared? Hov/ were persons participating in the 
study selected? What was the precise nature of the treatment given? 
What were the s^ges of those participating? What was the condition of 
their teeth before entering the experiment? What measurements were 
made? Who made the measurements? 

It is evident that we could make a long list of questions about 
this particular finding. But the important point is that the answers 
to these qtjestions have a great d^al to, do with how we interpret the 
results of the evaluation of different brands of toothpaste. 
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Let's ^take a closer look at these questions. You will note that 

some of the questions actually raise doubt as /to the truthfulness of 

the finding, Kdw sure are we that a certain brand of toothpaste was 

the factor which really made the difference in the number of cavities? 

Perhaps there were factors operating, other than a particular brand of 

toothpaste, whifch could help reduce the, number of cavities. For 

example, suppose the persons using the touted brand of toothpaste 

brushed their teeth three or four times a day; suppose great pains 

were taken to see that they cut down on their consumption of candy, 

coke, and other sweet foods; suppose only volunteers were used in the 

experiinent; or suppose boys and girls were selected for the experiment 

because they had a history of many cav^ities. Each of these factors 

could contribute to a reduction in the number of cavities regardless 

of the brand of toothpaste used; or at best, when^the use of a certain 

toothpaste was accompanied by these conditions, it may be more effective 

than when used under more everyday conditions, 
I 

So one of th^ crucial questions in interpreting evaluation findings 
is: Did the treatinent--in this case the use of a certain toothpaste— 
really make a .difference? Jf v;e can rule out or discount the other factors 
whfch may have caused or contributed to the result obtained, then we place 
a different interpretation on the findings than would be the case otherr 
wise. 

Before we leave this examfile, I call your attention to another' 
dimension of Interpretation hinted at by the skeptic's^questions. Even 
if we are satisfied that the finding is leg1timateC-that is, the brand 
of toGthpaste did rhake a di fference ,i n the experiment or study—parents 
could still wonder whether similer results will be forthcoming if their 
children usa the reconrmended br^nd of toothpaste. The point is that 
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valid findings 1n particular settings involving certain groups of people ^ 
at a particular point in time are not automatiCalty and completely 
applicable to what might be expetted to occur iti other settings with , 
different groups of people at some other time. So another crucial ques- 
tion in interpreting evaluation findings pertains to the degree to whi^h 
the findings can be generalized. . To be specific, the appropriate ques- 
tion is: To what groups, settings, situations, or types of measures do^ 
the results apply? 

I hope the example helps clarify and pinpoint two important questions 
that must be asked about evaluative studies designed to assess the outcomes 
of educational programs: Instead of the number of cavities; we are concerned 
with outcome measures such 3s change in behavior, ability to perform, and 
actual performance of persons who complete or leave vocational education or 
teacher education programs. Firsr, we need to assess the extent to which 
the outcomes observed can actually be attribute^! to the eduacational program- 
Did the treatment (educational program) really make a difference?. Jf our ; 
curiosity is satisfied on the first question, the second category of sus- 
pi cions probes this question: To. what gcoups, settings, situations, and 
types of measures do the results apply? - 

TO WHAT EXTENT ARE THE FINDINGS VALID? 
Some Preliminary. Considerations 

Before we examine this question in depth, we need to review briefly 
the comments made earlier about the intent, purpose^ or end sought for 
the research, Interpretatioji of the findings of rese5trch can only be 
made in terms of what the investigator set out to do. This morning I 
indicated that, in general » a particular research study can be placed in 
one Qf three categories that indicates the major purpose or intent of 
the investigation- I named those categories descriptive research, corr^la- 
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t1ona.l research, and experimental research. I'll Summarize the salient 

< 

features of each of these research strategies with the following , overhead 
transparencies-. 

* I. Descf^iptive Survey Research 

—end sought Is accurate description of a group or situation 

» — questions asked — 

what is the status^ of, ? 
what are th^, facts about...? 

—advantages 

* collect large amounts of data in a relatively efficient 
manner ^ \ 

" frequently a first step necessary for formulating priorities 
revising present programs, or developing new programs 

— limitations 

; - obtaining a . representative (unbiased) sample 

- obtaining" responses (non-response) 

^ - obtaining accurate responses (measurement error) 

— interpretation possible — description only 
IK Correlational Research 

— end soCight is the explanation of outcomes in terms of other 
factors, 

— questions asked — 

I wonder why, . . ? - 

How can I explain this outcome? 

What factors-are related to the outcomes observed? 

— major focus is to investigate relationships between variables, 
attempt to explain observed outcomes in terms of 

-T characteristics of participants 
characteristics of programs 
elements in the environment (situation) 

— advantages * ^' 

^ -- only feasible way of investigating problems of a certain 

nature - . . - 

helps identi fy iSosstble program interventions' 



2v 



20 

— limitations 



— biased sample; non-response; accurate response 
interpreting correlation as causation 



nterpretation possible 

association between variables 
- explanatory relationships between variables 



III, Experijnental Research 

' ^ — end sought is to "cause" certain outcomes to occur; test 
cause-effect relationships ^ . 

— questions asked — 

What will happen if,,.? 

How does program. A compare with program B? ^ 

" advantage— makes possible program assessment that allows 
interpretation that links, in a cause-effect mode, outcomes 
with certain processes 

— limi tations--requires consideration of two guestions 

Are the findings true (valid)? 

Can the findings be generalized? ' 
Comparison Groups ' 

Another important but related concern which cannot be ignored in ' 
interpreting the findings of evaluative studies is illustrated by the 
question: Wijth what group or groups was "our group" compared? In marty - 
instances evaluative studies of vocatipnal education programs do hot compare 
the performance of persons who complete a vocational education program or 
a certain pre-service or in-service teacher education program with the 
perfomance of persons who complete an alternative program. or, for that 
matter, with persons who complete no vocatiOfWtK^ucation or teacher 
education program. 

In one-group studies of this nature, we compare what are construed to 
be outcomes of the program with^hat we think the outcomes should be. 
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Actually, this type of evaluative study^is ^ case study that has as its primary 
intent the description of characteristics, capabilities, or performance of 
persons who have been exposed to the program. The evaluator who makes 
claims other than description for evaluation studies of this nature takes 
some rather dangerous risKs- 

A somewhat better alternative is. to compare enrollees' characteristics, 
capabilities, or level of performance after completing an educational 
program with corresponding measures before they enrolled In the program. 
This is the familiar before-after study fn which we compare enrollee? 
after completing an educational program with themselves before or at the 
time of enrollment^ Evaluators must be alert to factors that will be 
mentioned shortly which qualify. If not threaten, interpretations that can 
be placed on the findings of before-after stu(l|es. 

The major concerns In Interpreting evaluation findings are whether or 
not a comparison group is used and the nature and characteristics of the 
group or groups with whom the outcomes of vocational education programs 
ane compared. If we are to move beyond description with any degree of 
assurance that outcomes can be attributed to the educational program in 
which students participate, some provision must be made for the use of 
comparison groups , or control groups if you prefer the parlance of experimen- 
tation. 

Alternative Explanations for Outcomes Observed 

The main concern in evaluating educational programs is to determine 
what outcomes accompany or are produced by the programs. The research 
strategy is, to identify some factors which have the potential for affecting 
program outcomes that, in turn, might be mistakenly Interpreted as outcomes 
produced by the educational program. We are concerned with factors which 



22 

threaten the truthfulness (validity) of research findings. These possible 
thredts offer plausible alternative explanations to the hoped-for conclusion 
that the educational program produced (caused) the outcomes observed. 

Contemporary History (Current Events) , One possible alternative 
explanation has to do with happenings in the studer .s' environment* other 
than the educational program, which might contribute to favorable ratings 
on the criteria used to assess program effectiveness* The'risk is that the 
findings of th9 evaluation will be attributed to the educational program 
when* in fact, other events or experiences of students while enrolled in 
the program may have directly influenced outcomes. This possible threat 
to valid findings is labeled contemporary history* that is, events other 
than the educational program occurring from the beginning of the educational 
program to the time outcomes are assessed which may influence outcome 
criteria independently of the educatiorraKprogram- The threat of con- 
temporary history is real to a valid interpretation of findings yielded, by ^ 
evaluation studies which do not involve comparison groups> 

An example will help clarify. Take the case where the occupational 
perfomance of students completing a nonvocational high school curriculum 
is being compared with the occupational performance of high school students 
completing a vocational curriculum. Suppose a sizeable proportion of the 
nonvooational students had part-time jobs during the time they were enrolled 
in high school^-That work experience would surely influence their occupa- 
tional perfomance after graduating from high school. So that bit of con- 

A 

temporary history is a threat--an alternative explanation—to a finding 
indicating significant differences or lack of significant differences 
between the two groups in occupational performance* 

] ^ 
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General social and economic conditions influencing employment are 
an important part of contemporary history when vocational education programs 
are evaluated in terms of enrollees' labor market performance. This 
threat to valid Interpretation of evaluation findings 1s particularly 
applicable to data from follow-up studies* especially when the follow-up 
data are collected several years after students complete the educational 
program. Statements attributing occupational success solely to a voca- 
tional education program completed some three to^five years ear/ier are* 
without doubt, generally Invalid. Some investigators rightfully maintain 
that the effectiveness of ar> educational program Is most clearly seen In 
the first job held after graduation* for In subsequent jobs it Is difficult 
to differentiate the relative influence of the educational program from 
the influence of employment experience and other events occurring since the 
completion of the educational program. 

Normal Growth and Development . I propose that any program designed 
to teach children nine to eighteen months of age to walk will be highly 
successful. This illustrates another possible threat to a valid interpre- 
tation of research findings. It is obvious that most children between nine 
and eighteen months of age are going to learn to walk whether they partici- 
pate in a fomal training program or not. We must be alert to those 
situations where outcomes which we like to attribute to an educational 
program are not simply the result of normal maturation and growth of 
students. Maturation poses a threat that cannot bo overlooked in long- 
tem educational programs. Over a period of two or four years students 
are going to change considerably in physical, psychological, and emotional 
attributes which co4fribute to occupational success whether they are 
enrolled in an occupational education program or not. We must be on guard 
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not to attribute these normaHy occurring changes to outcomes produced by 
an educational program. For example* an evaluative study attributing 
eighteen- to twenty-year-old students' favorable attitudes toward work to an 
educational program could be questioned since it is about that age when 
many persons begin work to buy a car or establish a family and home. I 
suspect that natural growth and development is a more serious threat to 
valid evaluation findings in vocational education than many of us realize* 

Disregarding Persons Who Oo Not Co_mplete the Program . Frequently, 
evaluative studies only Involve persons who have successfully completed 
the educational program being assessed. If a considerable number of 
persons entering the program leave or drop out during the program* then 
it is obvious that an evaluation which Involves only those who successfully 
complete the program may give a less than accurate (valid) picture of the 
outcomes of the program. With few exceptions* it can be argued that persons 
w!io fail to complete the program successfully are al5o very likely to be 
persons wh^would be least successful in occupational performance. The 
point is that evaluative studies which only Involve students who complete 
a program run the risk of misinterpretation If there has been considerable 
mortality of enrollees during the conduct of the program. So outcoines ""'^ 
attributed to the program may"** in effect* be due In large part to the 
fact that measures are only made on persons^who are most capable as 
demonstrated by the fact that they completed the program. 

Measurement and Observation , We know that the results of evaluative 
studies' can be influenced by the type of measurements and observations 
used 3S well as by the persons who make the measurements and observations. 
In many respects it is difTicult* if not virtually impossible* for persons 
responsible for the conduct of a program to evaluate the progrlam objectively. 
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There is always the temptation to use outcome rneasures which we think will 
indicate success; there is always the temptation* when ratings involve a 
great deal of judgment* to error in the directrjn that Is favorable to the 
program* Generally* former enroUees have a tendency to respond on 
questionnaires or during interviews In a manner which they^percelve we want 
them to respond* which usually Is favorable to the program. 

You and I would probably be somewhat sur^plclous of the finding about 
the effectiveness of the brand of toothpaste if we tcnew the finding was 
the result of the manufacturer's research. He would be more likely to 
believe the finding if we knew the tesr. was made by an i^adependent testing 
agency. So it is wi^h '-valuation in vocational educatioru Let's be careful 
to design evaluative . tudies so that the outcof^es we cla-im to result from 
vocational education ^re not the' result of biased and incomplete measure- 
ment and observation. 

Extreme Cases , As a general rule, the more extreme we are on one 
measure today* the less (extreme we will be tomorrow or sometime in the 
future on the same or a related measure. For example* students who score 
extremely high or extremely low on a te^;t one da/* will tend to regress 
downward or upward on another test another day. One factor operating to 
produce that result is. statistical regression. There are twio situations 
where evaluators should be very sensitive to statistical regression as a 
possible threat to valid findings, .When educationa] prcgramo are provided 
students who score at the low end on criteria used in selecting participants* 
the students have no way to go but up, Resei>rcher$ must be careful net 
to credit this improvement in performance or achievement solely to th^^ 
effectiveness of the program for it is likely that statistical regression 
has been a factor also. The other situation involves evaluative studies 
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where vocdtlonj^l education students are matched with college preparatory 
students on variables such as academic achievement or I.Q. scores. 

if 

-Vi' ' 

statistical regression works in a very subtle fashion In these cases to 
produce* almost Invariably, significant differences between the two groups 
on subsequent tests regardless of the treatment received by students. In 
each group. 

Test-wlseness . By taking tests we learn how to achieve higher scores 
on subsequent tests. When evaluative studies involve testing of students 
before and during the educational program, students may be learning how to 
achieve higher scores, on tests at the end of the program because they become 
more test-wise. This tends to happen with attitude tests particularly and 
with achievement and performance tests when enrollees are not in art environ-* 
ment where testing is a common occurrence. Evaluati ve studies involving 
only one group of students where the same or similar tests are given before 
and after the educational program are particularly vulnerable to this possibl 
alternative explanation for a' finding that the program produced the outcomes 
described. 

How Students Are Selected . When outcomes of vocational education 
jjrograms are compared with outcomes of alternative programs* care must be 
taken to'avoid attnbutlng differences in program outcomes to differences 
in programs when* In f9ct, differences in outcomes may be Influenced more 
by differing characteristics of students enrolled than by differences in 

the nature* type, or duration of the educational program. 

— \ 

We know that there are characteristics of people that are positively 
related to performance* particularly performance-in the labor market,, 
regardless of whether an individual receives specific preparation or not. / 
Some of these characteristics include general level of ability, aptitude . 



, 27 
for certain areas of endeavor* interest* socioeconomic status* prior 

experience* and the like. So* if students electing vocational education 

courses differ on some or all of these characteristics from students not 

electing vocational courses or from students electing alternative avenues 

to occupational preparation* evaluative studies comparing the capabilities 

J or performance of persons completing vocatioVial programs with persons 
completing other programs are misinterpreted when differences or lack of 
differences in Outcomes are attributed solely to the nature of the educa- 
tional program. In these cases* the fact that students canpleting the 
programs have different characteristics offers a real and direct threat 
to the interpratation that can be placed on the evaluation findings. 
Differences or lack of differences in program outcomes could be influenced* 
or perhaps accounted for by differences between the groups of students on 
characteristics which are positively related to occupational performance. 

Generally* vocational education stands to gain more than it loses 
when this/threat to the validity cf findings is controlled or accounted 
for in evaluative studies . When compared to students in the academic 
curriculum* for example* vocational students usually rank lower on several 
of the characteristics (e .g . *geheral ability* certain aptitudes* socio- 
economic status) which are related positively to occupational success. So 
if the contributions of these factors to occupational success are. removed 
or accounted for in the case of students completing the academic or a 
nonvocational curriculum, logic supports the argument that the benefits 

"from vocational education may be greater than some evaluative studies 
have indicated. 

My poilJit is this; Evaluajtive studies comparing the outcomes of 
vocational education with alternative prtgrams that ignore or overlook 
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the fact that enrollees in the various programs may also differ have 
high potential for yielding findings which may not be valid. 

Differing characteristics of enrollees in educational programs being 
compare* are a real threat to the valid interpretation of evaluation find- 
ings, since the evaluator rarely has any control over which students elect 
what program, Enrollees ** self-select" themselves into vocatione^l education 
or some other program. The same attributes, interests, and aspirations 
which lead students to select or not select a particular curriculunr^ may 
also be the characteristics which enhance or impede occupational success 
independently--or in spite of— the curriculum in which they enroll. The 
"true experimenter** takes care of this problem through random assignment 
of students to educational programs which, within known statistical limits, 
achieve equality of groups prior to enrollment. Researchers in vocational 
education rarely have this option. 

The. question '*Are the research fir+dings true (valid)?** pertains to 
the internal validity of the design. I have listed and discussed briefly 
several types of threats to interpretations of research findings that 
indicate that the treatment (the program) makes a difference; These threats 
can also be labeled as plausible rival explanations for the outcomes 
observed, or if you prefer, what Abby calls *'other possible causes." I 
hope one strategy is clear. If research in education and vocational educa- 
tion is to be interpreted such that the findings are internally valid, t^yl 
\minimum requirement is that a design be used that involves a control or 
comparison group, or the researcher must resort to other strategies, 
particularly statistical techniques, to control or account for extraneous 
variables, 

3o 
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TO WHAT EXTENT CAN THE FINDINGS BE GENERALIZED? 
Once the interpreter's curlousity has been satisfied about the validity 
of research findings, the next concern has to do with the extent to which 
the findings can be applied (generalized) to other groups, situations, 
measures of outcomes, and times. The assumption here* of course. Is that 
the findings are valid, that is, the findings describe outcomes which we 
are reasonably sure resulted at least in part from the educational program 
being assessed. 

To What Gtx)ups (Populations) Do The Findings Apply? 

Characteristics of Students or Trainees . In generalizing findings 
about the >"elat7ve, effectiveness or ineffectiveness of certain educational 
programs, care must be taken not to assume that results produced when 
students with a particular set of characteristics are enrolled will apply 
to another set of students with different characteristics. Findings 
resulting from studies of persons with particular characteristics can be 
generalized only to populations of persons of which the persons studied 
are representative samples. 

Non response . A severe limitation to the generallzability of evaluation 
findings resulting from follow-up studies is nonresponse* This is par- 
ticularly true when data are collected through mail questionnaires* In 
many cases, a sizable number of persons fail to respond. Unless there is 
evidence to the contrary, it can be assumed that respondents differ sub- 
.stantially from those who did not respond, especially on some variables 
which are crucial to the outcomes measured. The result is that the findings 
cannot even be general i zed wi th any degree of confidence to all persons 
enrollins in or completing the educational program* 

3j \ 



c 

30 

To What Settinqst Situations^ and Measures <lo the Findings^ Apply ? 

Frequently, well designed studies are conducted in ,a particular 
school system. So the user of the findings must be alert to the tempta- 
tion to assume that a successful program in one situation will be similarly 

I 

successful in a completely different situation or setting. 

Test Sensitization . Results yielded by evaluative studies involving 
many tests before and during the educational program may not be applicable 
to situations where a massive testing program is not used. Tests prior 
to and during an educational program sensitize students to the content of 
the educational program. Similar results might not be forthcoming in 
situations where students are not sensitized to what is to come. 

Hawthorne Effect . We know that people react differently when they 
are aware that they are participating in a special progV^am or when their 
actions and activities are being monitored. It is not uncommon for ^ 
better designed evaluative studies' to accompany new programs and pilot 
pruyrams. In these cases, it is very likely that those participating 
in the program, teachers as well as students* are aware that they are 
being evaluated. Evaluative results produced in these situations and 
settings may not be representative of what could be expected in similar 
settings >*ith similar students and teachers who are not being monitored 
or are not aware that they are participating in a special evaluative 
effort . 

Multiple Treatment Interference . In generalizing the results of 
research in vocational education we also need to pay attention to the 
extent to which the persons (subjects) involved in the research have 
been involved in previous experimental or^ilot programs. If the treat- 
ment or progr^am that is being assessed is one of a series that the 
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students have been Involved in, the results*-even though they are internally 
valid— may not be generalized safely to situations where the students have 
not been involved in a series of experimental or pilot programs* 

De scription of Program * The interpreter of research findings is 
helped a great deal in applying results If the researcher has described 
in a thorough manner the nature of .the program offered* This enables 
the user of tht; findings to make some rather clear-cut decisions as to 
what other types of vocational education programs the results might apply* 

I can illustrate the point with an example. Assume that we have a 
valid study indicating that graduates of a high school vocational program 
are very successful in obtaining well Paying jobs for which they were 
prepared* To what extent would you apply that finding if you knew that 
a part of the program was an aggressive placement service cooperatively 
sponsored and operated by the school and the state employment service? 
That particular aspect of the program--a placement service—could very 
well be a main factor contributing to the high placement rate of 
graduates* Knowledge qf this sort abOut the nature of the program is 
extremely helpful, if not essential* for making decisions about the 
extent to whixh research findings can 6e general ized^i' ^ 

Outcome Measures , Care must be taken not to assume that. valid find- 
ings pertaining to, one set of outcome measures will also hold true for 
another set of outcome measures* For example, findings regarding the 
economic benefits of vocational education cannot necessarily be general- 
ized to finings that would result if noneconomic outcomes measures 
were used^as evaluative criteria. Researchers should desi^ studies which 
include measures pertaining to all appropriate objectives of the educa^ 
tional program or programs being assessed. 

3o • ■ 



32 

SUMMARY 

I have listed some possible alternative explanations for research 
findings which state or Imply that the outcomes do, in fact, result from 
the educational program being assessed* These factors* or threats to 
internal validity, can produce results which may be mistaken for outcomes 
produced by the educational program being evaluated* It behooves the ; 
researcher as well as the user of research findings to be aware of these 
possible threats to valid findings.. Research studies that do not involve 
comparison groups^are particularly vulnerable. 

In addition to. making decisions about the internal validity of re- 
search findings, the interpreter must also make sonie decisions about. the 
extent to which the findings can be applied to other groups, situations, 
measijres, and times, I have discussed some factors which should be con- 
sidered in making decisions about the generalizability of research 
findings. Researchers label the questions about generalizability as 
threats to external validity. 

We must not accept without question the findings of research ^ 
studies. We must not assume that just because we observe pertain outcomes 
accompanying educational programs that the outcomes always and completely 
result from— or are caused by--the programs. We must be aware of^ the 
threats to valid interpretation and generalization of research findings. 
If all we can do is describe the competence and performance of persons 
who have completed and left programs, let^s admit it and make no further 
claims. If we can show a high degree of relationship between outcomes 
and program inputs, let's try harder to see if there really are functional 
relationships. If we claim that a particular program produces certain 
outcomes, let's demonstrate that we are aware of and have attempted to 
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account for other factors that could produce or qualify the claims we make. 

The next time you read a report of. research in vocational education 
or education* ask these two questions: to what extent did participation 
in the program produce, the results observed? If you are satisfietJ on 
that score* then ask: To what populations* situations, measures, and' 
periods of time can the findings be applied? 
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SOME RESEARCH NEEDS IN VOCATIONAL EDUCATION 

by Dr. Robert Wannbrod " ; 

During this session, I want to discuss research needs in two areas 
of vocational education. First, I want to share with you some thoughts 
about research needs in the area of teacher education. Specifically, I 
want to relate that discussion to the strategy or concepts of competency- 
based teacher education. Next, I will comment concerning research ^ or 
systematic inquiry/pertaining to the evaluation of vocational education 
programs. Even a casual reading of the Education Amendments of 1976 
indicates that evaluation and accountability are major mandates in the 
new federal legislation for vocational education* 

The two major areas- -teacher education and evaluation of vocation'iil 
education— are not independent, , In fact, thqy are highly interdependent. 
I will not belabor that point for I think the connectio^HSetween the 
two areas of concern is evident. 

Research Needs in Teacher Education . . 

/' 

I couch my comments concerning research needs in teacher education 
in the context of competency- or performance-based teacher edu£at4oft; As 
background for the cofunents that follow, perhaps I should indicate what 
I;-see to be the rationale, strategy, and components of competency-based 
teacher education. It is from this point of view that I identify certain 
problems needing research by vocational educators as well as *y others 
in teacher education* * . ^ 

The basic rationale of performance-based teachfet* eduction is that 
teacher behayiors are closely associated with stu^nt achievement. In 
other words, teacher behaviors directly influence what. and how much 

A 
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students learji» how effi ciently^^students learn, students' attitudes toward 
learnifig, and so forth. The basic premise is that there is a direct 
relationship between process {teacher behavior) and outcomes (student 
achievement) , ^ 

The strategy of performance-based teacher education generally 
involves four basic components. 

First is the identification of those teacher behaviors that are 
thought to be major determiners of student achievement. These behaviors 
are labeled by various persons and groups as performance elements, 
pedagogical performance elements, behavior statements, competencies, 
skills, or tasks. Regardless of label, the strategy is to specify, in 
supposedly measureable terms, the behaviors teachers are to exhibit. 
Some proponents of competency-based teacher education refer to these 
factors as high inference variables, A part of this step of the process . 
is a determinatton of an acceptable level of mastery for the performance 
elements or competencies , 

— Next, the process requires that appropriate experiences be provided 
the prospective or practicing teacher so he or she can master the com- 
petencies, claimed to be necessary. At this step in Ihe process, emphasis 
is placed on modular instructional units, individualized instruction, 
micro-reaching, internship, field experience, and the like. There are 
indications that swne persons define competency-based teacher etjucatlon 
primarily in terms of the nature of learning experiences provided in 
pre-se("vice and in-service education programs. That Is, some claim they 
have a competency-based teacher education program if they are using * for 
example, micro-teaching^ individualized instruction, or a field-centered 
irvservice education program, I hope we in vocational education are not 
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bMying that narrow and inappropriate definition of performance-l>ased 
teacher education. 

The third component in the process is systematic appraisal and 
assessment of teacher performance in terms of the prespecified competencies. 
If mastery Is achieved, the teacher attempts additional competencies until 
sufficient competencies are mastered to warrant credentials certifying the 
teacher's qualifications to teach. If mastery is not achieved, the teacher 
participates in additional experiences designed to develop the desired level 
Of performance or, supposedly, the teacher is cycled out of the teacher 
education program, if a prospective teacher, or out of the school, if a 
practicing teacher. 

" The fourth and very essential component is the validation of the 
competencies in terms of student^hievement or outcomes. In other words, 
does it make ahy difference what behaviors a teacher exhibits in terms of 
what students know. and are able to do? 

I will take these four components and indicate what I see as fruitful 
areas for research. In reference to the first component--the identification 
of teacher competencies--my position is that we probably already have more 
competencies identified than we know what to do with. The problem now is 
making sense out of the lists of competencies we have rather than attempting 
to discover more. Probably the most frequently referred to list is the 380 
or 390 competencies-identified in the study conducted at the Center for 
Vocational Education at Ohio State University (Cotrell, Chase, Holnar, 1972J. 
That list is the basis for the instructional modules that have been prepared 
and field tested by Center staff, which I understand are to be released 
shortly. There ar^ other lists of competencies. ^ 

I propose that we devote time ajc^d effort to the analysis and synthesis 
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of these lists In an effort to identify those competencies that we have 
reason to believe are potent predictors of students' perfomance. In other 
words, let's see if we can identify from this vast list of competencies those 
that hold promise of being high inference predictors. Some of the competen- 
cies are trivial; others are so gross and broad that they are difficult if ,' 
not impossible to measure. For example* one competency that has been 
identified is that the teacher "demonstrate personal concern for the student 
and his/her family/' 

My question is how do we measure the degree of competence a prospective 

\ 

teacher or practicing teacher possesses for that criterion? Wheniwe consider 
measurement of teachers' level of mastery of the various competencies , we 
must be aware that there is a tendency to emphasize those competencies that 
are the easiest to measure. Chances are that the competencies measured most 
easily may be the factors that are the least important. 

So my first research concern is an analysis and synthesis of the 
competency lists with the goal of identifying a lesser number of highly 
important competencies that hold promise as potent predictors of student 
outcofnes. Then we can concentrate on the development of instructional 
strategies that aid teachers in achieving mastery for these particular 
competencies . 

Another fruitful area for investigation relating to the identifieation 
of teacher competencies is the possible interaction between the level of 
professional competence and the: level of technical* or subject matter* ^ 
competence possessed by teachers. Why would we expect the same instructional 
strategies to be equally effective for teachers who are highly competent 
technically and for teachers who are not as competent in their technology 
or subject matter? Maybe the teacher who is highly ccmpetent in the content 
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and skills being taught can use a different set of professional competencies 
than a teacher who is not as highly competent in the content and skills 
being taught. On the other hand* there may be little if any connection or 
interaction between professional and technical competence. My point is that 
with competency-based strategies we have been dealing almost excl^sivelyj 
with professional competencies with only a slight glance toward the technical 
competencies. I am suggesting that dealing with one side of the picture 
almost exclusively may not be a realistic approach. 

This leads to another component of the process* namely the assessment 
of the degree of mastery possessed by those preparing to teach or by those 
who are currently teaching. In essence, we are dealing with problems of 
measurement--problems that are the very crjx of the research orocess. How 
do we measure objectively, validly, and reliably, a teacher*s degree of 
mastery of the competencies that best evidence and theory indicate as being 
most essential to maximize student achieveme: t? If the strategy of com- 
petency-based teacher education is to be implemented, we will be required 
to pay special attention to the valid and reliable measurement of the extent 
to which prospective and practicing teachers have mastered the competencies 
we claim to be the teacher behaviors that influence what students learn^ 
The development and refinement of instruments and techniques for assessing 
level of mastery can be a major concern for the researcher in vocational 
education ^ 

Another major area of research in teacher education relates to the 
fourth component of the competency-based teacher education model— that Is, 
the validation of teacher competencies in terms of what students learn. 
This is the familiar process-product research. The research question 
■^^ing asked is; What is the relationship between teacher behavior (process) 
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and student achievement (outcomes or product)? One of the most extensive 
reviews of a series of these process-product studies Is a report by Rosenshlne 

i 

and Furst (1971) which Is published in Research in Teacher Education > a report 

of a symposium sponsored by tte American Educational Research Association. 

Their review of the research ^nabled them to Identify eleven variables-- 

teacher behaviors— that tend to be related to or connected with student 

achievement. However>. the flavor of their interpretation of the process- 

product research is indicated by this quotation: ■ 

"This review is an ac^ission that we know very little about the 
relationship between (teacher) classroom behavior and studeait 
gains. The results of these studies provide hypotheses upon 
which to build teacher training models." 

McNeil and Popham/( 1973) in their chapter on '^The Assessment of 

Teacher Competence'' in the Second Handbook of Research oji Teachinq > reach 

a similar conclusion: 

.evidence. . .suggests that practice has been seriously 
weakened by the false belief that there are scientific conclusions 
which correspond to good teaching. (We err when we) promote teach- 
ing skills that are approximately consistent with scientific con- 
clusions as If these skills were certain* confirmed answers about 
how a teacher should proceed. to effect desirable consequences in 
learners. Instead* such skills should be regarded as hypotheses 
to be tested." ^ 

These interpretations of the relationship between teacher behavior and 

student achievement are relatively mild compared to the conclusions reached 

by others who carefully analyzed the process-product studies reviewed by 

Rosenshlne and Furst. In a paper titled "The Myth of Performance-Based 

Teacher Education" presented at the 1973 Annual Meeting of the American 

Educational Research Association* Heath and Neilson (1973) concluded that: 

"an analysis of the research on the relation between teacher 
performance and student achieveruent fails to reveal an empirical 
basis for performance-based teacher education. (The) literature 
fails to provide such a basis> not because of minor flav/s in the 
statistical analyses^ but because of sterile operational defini- 
tions of both teaching and achievement* and because of funda- 
mentally weak research designs." 
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Then they ir^aily drop the bomb: "...gtven the well-documented, strong 
assoclatlpn between Student ^chleventent and variables such as socioeconomic 
status and race (here they refer to Coleman's research), the effects of 
techniques of teaching on achievement, as these variables are conventionally 
defined, are likely to be inherently trivial/* 

Needless to say. If we are going to build a competency-based teacher 
education program, research investigating the relationship between teacher 
behaviors (process) and student achievement (outcomes) Is a necessity. We 
need to Investigate what behaviors tend to be accompanied by certain outcomes* 
This research Is complicated and difficult to conduct. is clearly multi- 
variate. The research probably will have tp^Degin with correlational designs, 
but if we are going to find out with any degree of certainty what teacher 
behaviors make a difference, eventually quasi-experimental and experimental 
designs will be requir^. This research requires that we accurately describe 
and quantify not only what behaviors teachers exhibit, but also that we 
validly and reliably fnejasure what students have achieved. Perfips an avenue 
to pursue in assessing students' level of achievement or mastery is what 
Popham (1973) at UCLA calls "teaching performance tests", which, in effect, 
are criterion- referenced tests designed to assess students'^ level of 
mastery in terms of the learning outcomes specified in the objectives for 
a particular unit of Instruction, 

So to summarize briefly, I propose that the so-called competency-based 
approach to teacher education highlights several priorities for research. 
First, is the analysis and synthesis of the vast array of teacher competencies 
that have been developed in an effort to refine the list to a manageable 
group of high Inference teacher behaviors that appear to be potent pre- 
dictors of student achievement- Next is the problem of measurement and 
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Instrumentation. Specifically, the problem is the assessment in d valid and 
reliable fashion of both teachers' level of mastery of certain competencies 
and students' level of achievement of what is taught. And finally, there 
is the problem of conducting the difficult but all-impoicitant process-product 
research in an attempt to validate teacher competence in terms of student 
achievement. 

Assessing the Effectiveness of Vocational Education 

— ! ^ 

For the remainder of this presentation, I focus on another aspect of 
needed research in vocational education* That is systematic Inquiry designed 
to assess the effectiveness of vocational education programs* A reading of 
the Education Amendments of 1976 (Public Law 94-482) makes It deaf that the 
writers of the .legislation expect their mandated evaluations of vocational 
education programs to yield data and information useful in formulating on- 
going and future policy and program decisions. I have recently had called 
to rny attention a document that lists 23 separate studies and reports man- 
date^ in the Education Amendments of 1976. 

Let me set the stage by reviewing briefly some of the provisions of the 
legislation mandating the evaluation of vocational education programs. In 
order for the federal government to assist states, the Bureau of Occupational 
and Adult Education Is charged to "conduct a review analyzing the strengths 
and weaknesses" of vocational education programs in at least ten states each 
year. States, in turn, are directed to evaluate the effectiveness of voca- 
tional education programs in providing assistance to local education agencies. 
This section of the legislatiQn specifically directs each state to evaluata, 
by using data collected whenever possible, by statistically valid sampling 
techniques, each program which purports to impart entry level job skills 
according to the extent to which pinogram completers and leavers (i) find 
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employment In occupations related to t|ie1r training, and (ii) are considered 
by their, enfl)1oyers to be well-train^d and prepared for employment. Other 
provisions of the legislation dlrect^the National Institute of Education to 
undertake "a thorough evaluation and stuc(y vocational education programs", 
^including a review and evaluation of home economics programs* 

The language of the act clearly Indicates that the results of these 
evaluations are to be used to make recommendations for the revision,, re- 
direction, and improvement of programs and in formulating. recommendations 
for changes In existing legislation and for new legisl^Ton* My argument 
is that valid and reliable data required for formulating policy and program 
decisions will not be forthcoming unless evaluation studies of vocational 
education are planned and conducted in as rigorous Itnd^disciplined a fashion 
as can be mustered. ^ 

To date* those who have studied evaluation studies in vocational educa- 
tion have been less than complimentary about the methodology, design, and 
rigor of the research. Moss and Stromsdorfer' s (1971) review of both economic 
and noneconomic studies of vocational education led them to conclude some 
five years ago '*that the methodological issues facing any analysis of the 
effects of vocational education are fonnidable, • .that the current^quality 
of evaluation can be raised significantly by careful attention to proper ^Jse 

of methodology .that the existing analyses, taken as a who]e, do not use 

to 

effectively the methodological knowledge which is currently available*" 

Professor Reubens (1974) of Columbia University, who authored a paper 
for use by the Special Task Force to the Secretary of Health, Education, and 
Welfare in preparing what to some vocational educators was the controversial 
report Work in America , had this comment about evaluation studies of voca- 
tional education programs: "The assessment of current high school vocational 
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education Is beset with conceptual programs* methodological pitfalls,- 
and statistical limitations. It Is not unusual for Investigators to devote 
more space to discussions of these issues and criticism of other studies 
than to the presentation of their own findings." 

The Committee on Research and Development in Vocational Education 
(1976) is sharp and to the point in its appraisal of evaluations of voca- 
tional education programs. Here Is a safnple of their rhetoric: "^Research 
on sume important and difficult issues, such as measuring the actual 
benefits of vocational education, is nearly nonexistent. The literature 
describing the evaluation of vocational education programs is discouraging* 
it yields little useful infomation for vocational educators. The research 
designs have used analytic procedures requiring simple quantitative input 
and have failed to encompass many important educational issues. Evaluations 
have used research methods that are incompatible with the complexity of the 
l&arnin<^. teaching* and administrative situations*" The Cofumittee on 
Research and Development in Vocational Education of the National Research 
Council, Nation|l Academy of Sciences, is a distinguished group of voca- 
tional educators and social and behavioral scientists, chaired by Rupert N. 
Evans, that has recently released its assessment of the research and develop- 
ment activities sponsored by the U. Office of Education under the pro- 
visions of the Vocational Education Act of 1963 and the 1968 Amendronts* 

In view of the foregoing, it appears that those of us who have interest 
and expertise in research in vocational education face a difficult and 
perhaps Insurmountable task if we accept the challenge of improving the 
Interpretabili ty and generalizabiHty of disciplined inquiry i^to the 
effectiveness of vocational education. ^Should we give up or try harder? 
I suggest we opt for the latter, I will sketch the broad outlines of what 
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I see as some major concerns that merit attent16n as we attempt to apply 
methodological rigor to evaluation of Vocational education. 

Since I have reported how several persons and groups appraise the 
methodology of evaluation research in vocational education, perhaps I - 
should summarize briefly the general findings reported Ijy those who conduct 
and review these studies. The Special Task Force to the Secretary of 
Health, Education, and Welfare (1975), apparently drawing heavily from the 
resource paper prepared by Reubens (1974) that I cited/earlier, concluded 
that "vociitional education in the high schools has failed to give students 
useful skills or place them in satisfying jobs." Somers (1971) of the 
Center for Studies in Vocational and Technical Education at the University 
of Wisconsin, who studied a national sample of vocational students three 
years after they had graduated from high school, branded as. a "half-truth" 
the i»otion.that vocational education is designed to prepare people for 
entry into the world of work. 

Grasso (1975), using a^ational sample of male youth from the National 
Longitudinal Surveys^ a U: S. Department of Labor project conducted by the 
Center for Human Resource Research at The Ohio State University, concluded 
that an analysis of the relationship between career preparation of youth 
and a wide variety of career-relevant performance criteria fails to support 
the case that vocational education is superior preparation for the world 
of wor-k for male high school graduates who do not go on to college, Some 
vocational educators are still haunted by Wilms* (1974) headline-making 
claim "that. eight out of every ten graduates of professional and technical- 
level, post-secondary vocational programs did not get the jobs they trained 
for ..." 

Now we have two interesting sets of generalizations that, at best. 
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create a dilemma for those who make a serious effort to use evaluation 
research in policy and program development in vocational education. On 
the one hand, assessment methodology is described as rarely adequate, 
beset with methodological pitfalls, and as yielding little useful informa- 
tion. On the other hand, the results of the more extensive and widely 
quoted studies fail to. provide empirical support for vocational education 
as preparation for employment, particularly at the high school level. 

A major concern that must be dealt with, if evaulation research Is to 
result In valid and reliable data for decision making has to do with the 
selection of appropriate criterion variables, ^ince the outcomes desired 
are derived from the objectives of the program, it is Important that there 
be some agreement on what is to be achieved through vocational instruction. 
Until there is some degree of concensus on goals, there Is not likely to 
be a great deal of agreement on the appropriate criteria for assessing 
outcomes. 

A part of this concern is the issue of whether vocational education . 
and general education are considered to be mutual contributors to common 
educational goals, or whether we perpetuate the vocational education-general 
education schism that assigns preparation for work to one segment of the 
curriculum, preparation for life to another segment,^ and for some senseless 
reason, assumes that one has little influence on the other. In a previous 
paper, I (Warmbrod, 1974) have argued that the separation of vocational, 
and general education at least partially accounts for the recurring 
appraisal thatSocational education is less than successful in accomplish- 
ing the outcomes claimed by Its proponents or in achieving the challenges 
posed by its antagonists. Two limiting assumptions accompany this sepa- 
ratism. The first is the view that the development of occupational skills 
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is the exclusive bailiwick of vocational education; the second is a failure 
to recognize that general education sitills are major contributors to ijccu- 
pational competence and success. These two restrictions fairly well scuttle 
the pnjspect for discerning the extent to which a specific segnient of the 
curriculum, be It general education or vocational education^ impacts on 
students* post--school behavior*^ So, evaluations skeptical of the effec- 
tiveness of vocations] education in achieving a narrow goal of occupa- 
tional proficiency arfe practically assured^ if it is expected that specialized 
vocational skills are so pervasive as to override all other factors that 
influence occupational success. Surely^ general education skil Is/ socio* 
economic and personal characteristics of students^ economic and labor market 
conditions^ and employment practices are among the factors that must be 
considered when studying the effectiveness of vocational education programs. 

The Coimittee on Vocational Education Research and Development (1976) 
contends, that criterion measures inadequately reflect program success when 
outcomes are measured in terms of initial job placement^ which in turn is 
Influenced by factors such as the availability of jpbs» social status^ per* 
sonallty and intellect of the stu^ent» and chance. The committee proposes 
that additional relevant criterion variables might include job turnover 
rates» job satisfaction^ the socioeconomic mix of students^ and changes in 
students' self*perceptions, Others have proposed that appropriate criteria 
for evaluating vocational education programs include work habits^ values^ 
and attitudes? 

Perhaps the most direct way to pinpoint the crux of the issue I wish 
to highlight is by posing 'this question: Should vocational education be 
evaluated as a function of the total school program* or as the unique and 
exclusive function of one segment of a school's program? Stated in a 
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slightly different form the question becomes: Is the development of 
occupational competence in its brq^dest sense among the outcomes sought 
from the total school progranit or is occupational competence the exclusive 
domain of those courses and experiences that are' labeled "vocational'*? ^ 

, The approach one takes to assessmentt including the selection of 
appropriate criterion variables* hinges on the answer to these questions. 
If the former perspective is taken, there is a gamut of criterion variablest 
ranging from specific skill competence to attitudes and valuest tbat are 
appropriate for assessing the extent to which the total school program is 
achieving its goal of occupational preparation. If the latter perspective 
is takent the range of legitimate criterion measures becomes more narrowt 
consisting primarily of an assessment of the extent to which the specific 
occupational s-kil Is taught in vocational courses are acquired. To expect 
one segment of the curriculum— instruction and experiences in courses 
labeled vocational — to be the major or sole factor that detennines whether 
students get jobs or not* the lievel of the jobs obtained* wage level » job 
satisfaction* human relations skillst self-perceptions* attitudes* and 
work habits is an unreasonable* if not naive* position- To be blunt about 
it* we in vocational education have helped build a system that assigns 
accountability for the school's occupational preparation goal almost ex- 
clusively to what we ten^ the vocational segment of the curriculum when* 
in fact* all parts of the school's programs-academic as well as vocational-- 
have a stake in achieving the goal of education for the world of work- In 
effect* the separation of general and vocational education that I referred 
to earlier— fostered* incidentally* in many cases by vocational educators 
and the policies they develop and administer— in many respects lets the 
total school program off the hook when it comes to accountability concerning 
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the schooTs goal of occupational preparation. As long as vocational educa- 
tion claims to be the sole input lor the knowledge, skills, and attitudes 
required for occupational competence, it will be vocational education! not 
the total school program, that takes the brunt of criticism that sch^ls ^ 
are not accomplishing satisfactorily their functions of occupational develop 
me^nt and preparation. . ; . ^ 

How does this discussion, of vocational education as a function of the ' 
total school program versus vocational education as the exclusive domain 
of a segment of the school program relate directly to evaluating the 
effectiveness of vocational education? The connection is clear and direct. 
When vocational education is viewed as a function of the total school 
program, some important issues relevant to the design and conduct, of evalua- 
tion studies come into focus. First, the target population becomes all com- 
pleters and leavers of the school, not limited to those who complete or 
leave a vocational course. Second; the levels of the treatment in the 
evaluation design become the various mixes of vocational education and 
general education that students experience. This recognizes that all 
segments of the curriculum contribute to the occupational education functipn 
and will make it possible to investigate what segments of the curriculum 
contribute to specialized occupational skills and to the more general occu- 
pational skills of communication, computation, and human relations; And 
third, when vocational education is viewed as a /unction of the school, a 
variety of criterion .measures pertaining to occupational development, 
decision making, and corppetence become appropriate. In effect, an eijjilua- 
tion of the school's occupational education goal becomes a part of the 
evaluation of all the goals and functions of the school. 

In this context, evaluation of a school's occupational education 
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function becomes a complicated and methodologically sophisticated affair. 
Perhaps an,;|ippropriate way to describe the evaluation is something of the 
magnitude of the Coleman Report (Coleman et al., 1966). 

^ly point Is Jffi^ this type of evaluation is called for if the results 

are to yield Information that makes a maximum contribution to policy and 

/ 

program development. In this context, evaluation of vocational education ^ 
is seen as the description and explanation of a series of outcome variables 
in terms of a variety of input variables--in1tial abilities and motives 
of students, family and community characteristics and values, school re- 
sources and policies* peer groups,, and the nat^jre of the instructional 
program and experiences of students In school and out of school. In addi- 
tion, when the criterion variables relate to labor market entry and per- 
formancej W}en another set of factors over which the school has little, if 
any, control comes into play— economic conditlops, employment practices, 
mobi^lity and migration of students, for example. ' 

The evaluation studies being described are difficultand expensive to 

/ 
/ 

conduct. see such studies being commissioned by the U. S. Office of 

EducalU^n, the National Institute of Educaf^tion* and states where there is 

interest in a thorough appraisal of the extent to which schools are accomp- 

lishing the occupational education goal. The research methodology is 

probably most accurately described as a naturalistic correlation method ' 

\ 

with regression analysis and other multivariate techniques the major 
statistical strategies for attempting to partial out the influence of the 
various inputs on the criterion measurS's. In evaluation research of this 
type* generaTizability of results is probably accomplished more readily by 
replication than by random selection of subjects from a state or national 
target population. " ^ 
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A realistic approach to evaluation of vocational education does not 
begin* however* with the type of evaluation studies I have described* We 
might as well face the facts* we must begin where we are, ■ I doubt if I 
will get much argument with the statement that most current endeavors 
purporting to evaluate vocational education fall under the general heading 
of follow-up studies. In fact» the provisions of the new federal legisla- 
tion requiring data concerning the extent to which program completers and 
leavers find employment in occupations related to their training and 
whether they are considered by their employers to be prepared for employ,- 
ment indicates the continued importance that will be placed on follow-up 
studies. ) 

In research parlance* most follow-up studies are conducted as one-shot 
case studies— the exemplar of a lack of methodological sophistication ff 
one wishes to ascribe outcomes to the treatment factor* We need to realize* 
and decision-makers need to realize* that most follow-up studies, at best* 
only provide descriptive information. I do not propose that we abandon" 
follow-up studies. Instead* I suggest we clean up their most flagrant de- 
ficiencies and build upon them to design evaluation studies that yield more 
valid information for the decision-maker* Since most follow-up data are 
collected by mail questionnaire* the resulting data will» in most cases* ^ 
be questionable unless problems of measurement error* reliability of ques- 
tionnaire items* and bias due to nonresponse are considered and dealt with. 
For example* Project TALENT researchers have found that conclusions based 
on data provided only by mail follow-up questionnaires cannot be sa'fely 
generalized to the population from which the sample was drawn because of 
ti\e selected nature of the respondent group with respect to academic 
aptitude and family characteristics such as socioeconomic status 



(Flanagan et al., 1964), Investigation is needed on the concurrent 
validity of follow-up data generated through mall questionnaires. In other 
words, how do we know people are not lying to us? 

it 

One way to Improve follow-up studies Is to follow up all school. com- 
pleters and leavers. If that strategy is contained with a description of 

J 

the nature of the educational program or curriculum pursued by the completers 
or leavers, then we are well on the way of moving from description only to 
that of explaining variability in post-school competence and behavior in 
terms of the general education-vocational education mix experienced by 
students. At this step of the process, it is ""important that the treatment 
variai)le--type of educational program—be precisely rather than grossly 
defined. The most common definition is to categorize students as vocational, 
academic, ^and general < These broad categories have a great deal of overlap, 
so mojre precise definition and categorization is called for. In fact, 
jmore precise definitions within the vocational category are essential. 
There are vast differences among vocational programs in the various occupa- 
tional areas, differences between programs involving supervised on-job ex- 
perience and those that do not, and differences between programs in compre- 
hensive hijgh schools and separate vocational schools. If additional input 
variables vor other parts of thie, system are added to the model, further 
progress is made toward the more complete naturalistic correlational evalua- 
tion strategy that was discussed earlier. 

The tasks .of evaluation in vocational education set before us are 
difficult, time-consuming, and expensive. But we do not have to begin from 
ground zero. There is evaluation research in vocational education in which 
we can build. There is a vast array of expertise and experience in evalua- 
tive research in other "areas of education and social and behavioral science 

er|c 
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(Glass» 1976; Struening and Guttentag, 1975), For example, the report of 
a conference held at the Brookings Insti uUtfon concerning evaluations of 
the Head Start and follow-through programs, published under the title 
Planned Variation in Education , treats several of the problems that are 
encountered in sophisticated evaluation designs (Rivlin and T1mpan&, 1975), 
A basic reference may well be the recently published book by Coo ley and 
Lohnes (1976) titled Evaluation Research in Education: Theory, Principles, 
and Practice . 

We need to explore the possibilities for further investigations 
through secondary analysis of existing data banks such as Project TALENT, 
the National Longitudinal Surveys sponsored by the U, S, Department of 
Labor, the National Longitudinal Study of the High School Class of 1972 
(National Center for Educational Statistics, 1976), and the National Assess- 
ment of Educational Progress. The reporfej^f the First National Assessnient 
of Career and Occupational Development were released in November (National 
Assessment of Educational Progress, 1976), The next Assessment of Career 
and Occupational Development will be made in 1979*80, Why don't we explore 
the possibilities of building into the 1979-80 assessment additional infor- 
mation that is needed for a comprehensive, nationwide assessment of voca- 
tional education? The national vocational education data reporting and 
accounting system that is required by the Education Amendments of 1976 
may well be a valuable source of information for specific evaluation studies, 

I have attempted to highlight some of the major concerns demanding 
attention if decision-oriented inquiry is to yield relevant and valyd 
information for use in policy and program development in two areas— teacher 
education and evaluation in vocational education. I propose that these 
concerns are and will continue, to be major research needs in vocational 
education. 
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^lELD TESTING THE COMMON CORE CURRICULUM MODULES 
^ by Dr. Terry G. Newell 

The^pdmary goal of the Common Core Curriculum evaluation for 1976- 
1977 was to field test the newly developed modules. Perhaps "field review" 
might be a better description of what was actually accomplished since the 
term "field test" usually implies the use of an experimental or quasi - 
experimental design. Those designs usually require the use of a control 
group which receives a traditional approach or treatment; the method used 
for the field tests reported here was a simple pre-post design with no 
control group. 

Field Test Procedures .and Materials 

During early December, 1976, a letter requesting expression of interest 
in participating in field testing of the Commor\ Core modules was sent to' 
all teacher educators in Vocational Education at 20 colleges and universities 
in California. The teacher educators, in-turn, were asked to contact any 
colleagues who might be interested in field testing the modules. Accompany- 
Ing the letter was a list of the modules and a short description of the 
contents of the modules, (See Table I.) 

After further communication, a total of 66 field tests were tenta- 
tively arranged. The modules most in demand were those in Series V; 
Curriculum Design in Vocational Education; Module V-b (Applying Learning 
Theory) was requested seven times and Module V-a (Curriculum Design) was 
requested five times. Each module was requested at least once. Individual 
instructors proposed to field test anywhere from one to five modules.^ 

Ultimately, 35 of the 66 field tests (53%) were completed. These 35 
field tests utilized 24 of the 29 newly developed modules and included 
students from the areas of Business Education, Home Economics Education 
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and Industrial Education, The most common reason for the noncompletlon 
of field tests seemed to be that the instructor did not have enough class 
time to use the module or was unable to incorporate the module into the 
content of a specific class. In a, few cases , the class for which the 
field test was proposed did not reach sufficient enrollment for it to be 
taught. 

Upon confirmation that an instructor wished to field test the TTK)dule(s), 
a letter containing instructions, the instructor and student versions of 
the modul?, and a mMs of ancillary material was distributed. The ancillary 
material included (1) a content validity questionnaire, (2) Student Opinion 
and Satisfaction Questionnaires, (3} Student Background Data Sheets ahd 
(4; a Student Progress Check List, The instructor was to administer and 
score the pre and post tests and was to monitor the progress of the student 
through the module. The specific manner in which the module was utilized 
was left completely to the discretion of the instructor. 

The next section of this report summarizes the field test results of 
two modules. Ti make scanning of the tables, the summaries of the student 
opinion questionnaire and the pre-post tests, somewhat easier, any mean 
of 3.5 or more and any mean of 2,5 or less, 1$ circled. The reason for 
this is that a rating of 3,0 indicates a "middle-leveV* or '^acceptable", 
attitude or opinion about the aspect of the module covered by the item. 
It should be noted that high ratings are not necessarily *'good*', nor are 
low rating^ necessarily "bad**. Each item has its own anchor points and 
these anchor points are briefly identified in the table. Additionally, 
any standard deviation of 1.0 or more is circled to indicate that there 
is a fair amount of diversity in the responses to the item. 

For one of the modulesi the mean content validity ratings of the 



module are included. Two consultants rated each module and these and any 
other content validity ratings obtained at the CCC Workshop were given 
to the appropriate module auttjor to aid in the revision of the module. 
Field Test of Module Ill-g r Student Organizations 

The field test of Module Ill-g serves as a model of what a well- 
motivated, competent, interested instructor can do with the Common Core 
approach. A class of seven graduates and four undergraduates in Home 
Economics Education covered the rrtodule over a six-week time period. The 
instructor slightly modified the module to meet the specific needs of her 
class and then provided each student with all the necessary reference 
material needed to complete the module. The instructor chose specific, 
activities for each lesson which were appropriate for her class and could 
be reasonably accomplished given the tiine and resources available; The 
students, working i ndivi dually. andj in small groups, completed all these 
activities. The instructor kept a good record of the progress of each 
student, evaluated the quality of the activity and then gave feedback 
to the student concerning their performance. _y 

The instructor's well-designed use of the module was reflected in 
the students* opinion .of the module (Table 2). They felt mastery of the 
objective was important; the references were easily available; they liked 
the module approach and rated the student-instructor relationship as 
excellent. They did think the module was somewhat too long and there were 
too many .activities, but they also felt that ha\/ing Completed the module, 
they could meet the behavioral objective. 

The instructor's well-conceived use of the module was also reflected 
by the impressive gain in scores from pre to post test. Starting from a 
very low mean of 11.0, the-, students gained an average of 76.1 points. 
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y1eld1ng^a post-test mean of 87.1, This significant increase (t(10) = 
31.77, p< ,01) coupled with the relative homogeneity (S = 6.7) of scores 
on the post-test/should be construed as positive evi dence of the potentia 
usefulness of the Common Core approach and furthermore, provides a goal 
toward which future field tests may aspire. 



Table 2: Summary of Student Opinion and Content 
Validity Ratings for Module Ill-g; (N=ll) 
Student Organizations 





Student 


Opinion 


tontent Validity 


Item; 


lean 


Standard 
Deviation 


Mean 


1 


nuw iiripur uaii u 15 iiiaS uery UT tne 

module objective; 

Very Not at all 




0.6 




2, 


Organization: Excellent 

Poor 


3.3 


0.7 




3, 


Length: Too long,,, .Too short 


s 


0.7 




4. 


Vocabulary Level: Difficult 

tasy 


3,2 


0,4 




5. 


Refereinces Used: Readily 




0 8 




6, 


Activities: Too many,,, 

Tnn "Fau 

1 UU 1 CfV 




I 0 




7, 


Pre-Post Test: Too difficult.,. 
Easy 


3,4 


0-8 




8. 


How well do you meet behavioral 

objective; Very well 

Not well 




0,4 




9. 


How did you like module approach? 


3.3 


0,7 




13. 


How closely did instructor keep 
track? Very close Rarely 




0.6 


f ■ 




Student-Instructor Relationship 




0.6 ^ 





Table 3; Results of Pre and Post Test 
for Module Itl-g: {N=ll) 





Mean 


Standard Deviation 


Pre Test 
Post Test 
Difference 


U.O 
87.1 
76,1 


8.9 
6,7 
7,9 




t(10} =3 


1,77 , p<,01 
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Field Test of Module V-a : Developing a Curriculum Design for Vocationeil 
Education . 

Module V*a was field tested at three sites. At site one, five 
graduate students in Business Education were given the module and told to 
COT^plete as much of it as possible over a two-week period during which 
the instructor was absent from class. The instructor provided no help to 
the students. The individual accomplishment of activities was not recorded* 
Table 4 summarizes the opinions of four of these students. They thought 
the module was too long, the vocabulary level was too high, there were 
too many activities and the test was too difficult* The references were- 
not available and they felt they coulu not meet the behavioral objective 
very well. Analysis of pre-post scores .indicated a modest but significant 
{t(3) =4*49, p<.01) average gain of 14 points* (Tables) f 

Field testing at site two provides a marked contrast to that at site 
one; here, eight graduate students in Business Education studied the module 
over a five*week period. They did one lesson per week, arid the instructqr 
monitored the students* progress to some extent. Students* opinions of 
the module (Table 6) were similar to those at site one; the module was- too 
long, there were too many activities and the test , was difficWt, but re* 
ferences were now available and they did seem to like the module approach 
somewhat more than the students at site one. There wa^ relatively little 
variability^ across the opinion items, indicating that the students tended 
to have similar responses to the module. 

Pre-post scores (Table 7) showed an average increase of 42 points 
(tE7) =8.07. p<.01). 

At site three, six graduate students in Business Education worked 
on the module over a five*week period. Each student completed one activity 
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per lesson and the Instructor evaluated the quality of the activity and 
kept track of the students' progress. 

Student opinions {Table 8) of the module Indicated that. In contrast 
to the other sItesV they thought the module was well-organized, but they 
agreed with students at the other sites that the module was too long and 
the test was too difficult. Overall* these students liked the module 
approach . 

On the test* (Table 9} these students gained an average of almost 51 
points (t{4) =15.5. p<.01}. Since all three sites utilized graduate 
students in the same area, and the pre-test scores of all three groups 
were similar, the Increased gain in post-test scores at sites two and 
three is probably attributable to the Increased instructor effort at these 
sites , 

Content validity was assessed by five raters; they were favorably 
impressed with the module though they Indicated that the references 
might not be readily available and that the test was jJIfficuU. ThVy 
felt that a student completing the module would meet thQ behavioral 
objective. 

. . r ' . ' , . ■ ■ 

/ ■ 

ERIC ^ ^ 



Table 4; Suimary of Student Opinion and Content 
Validity Ratings for Module V-a: (N=4) 
Developing a Curriculum Etesian for Vocational Education 





Student Opinion 


Content Validity 


Item: 


Mean 


Standard 
Deviation 


Kean 


K How important is mastery of the 
Not at all 


3.8 


0.8 




2, Organization: Excellent 
Poor 


{2j) 


(ij). 




3. Length: Too long Too short 




• 0.0 




4, Vocabulary Level: Difficult*,.. 
Easy 




0.0 




5. References Used: Readily Avail* 
able..,;. Not Available 




0.5 




6. Activities: Too many.*.*Too few 




0.5 




7. Pre-Post Test: Too difficult... 
Easy 


© 


0.0 




8. How well do you meet behavioral 
objective: Very well Not well 


© 


0.0 




9. How did you like module approach? 
Very much*,*, Did not like it 


©■ 


0 




13. How closely did instructor keep 
track? Very close Rarely 


© 


-.0.0 




14, Student- instructor Relationship 
Excellent Poor 


3.0 


e 





Table 5;' Rpsults of Pre and Post'Test 
\ fry,- Module V-a: (N=5) 





Mean 


Standard Deviation 


' ■ — 

Pre Test 


23.8' 


9.8 


Post Test 


52.4 


13.7 


Di f f erence 


28:6 


14.2 



t(3) =4.49 , p<.01 



Table 6; Summary of Student Opinion and Content 
Validity Ratings for Module V-a: (N=8) 
Developing a Curriculum Design for Vocational Education 





Student 


Opinion 


— 

Content Validity 


~~ ■ r 

Item: 


^ean 


Standard 
Deviation 


Mean(N=5) 


L. 


H6it important is mastery of the 
Not at all 




i 

0,7 V 


5.0 


2, 


Organization: Excellent 

Poor 


0 


0.7 


4.0 


3, 


Length: Too long Too short 


0 




3.0 


4. ■ 


Vocabulary level : Difficult 

Easy 


3.1 


0.8 


3.2 


5. 


References Used: Readily Avail- 
able, , , , Not Available 


2.7 


0.9 . 


2.4 


6. 




(O) 


0.7 ; 


3.6 . 


7. 


Pre-Post Test: Too difficult..- 
r AC V/ 




o:^ 


3.8 




How well do you meet behavioral 
objective: Very well. ...Not well 


(Q) 


0.9 


4.2 


9. 


How did you like module approach? 
Very much Did rrot like it 


3.1 


- 0.9 




13. 


How closely did instructor keep 


0 


0.5 




n. 


Student-Instructor Relationship 


0 


0.4 





Table 7: Results of Pre and Post Test 
for Module V-a: (N=8) 



r 


M^an 


Standard Deviation 


^re Test 


25.5 


12.9 


Post 'Test 


67.3, 


13.5 


Di fference 


42.2 


14.8 



t(7) =8.07 , p<.01. 
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Table 8*. Summary of Student Opinion and/Content 
Validity Ratings for Module V-a: (N=6) 
Developing a Curriculum Design for Vocational Education 





student Opinion 


Content validitv 


Item; 


^ean 


Standard 
Deviation 


Mean 




now 1 inporiani is wiah irsry ot , ine 
Not at a f 1 




fi 7 




2. 


ruur 


(G) 


0.8 




3. 






U. 0 




4. 


Easy 


J* 1 


U. H 




5. 


Referencer Used; Readily Avail- 


J. c 


u. y 




6. 


Activities: Too many... .Too few 




0.5 




7. 


Pre-Post Test: Too difficult... 
Easy 




0.9 




8. 


How well do you meet behavioral 
objective: Very well.. .Not well 


3.0 


0.8 




9. 


How did you like module approach? 


3. 1 


0.4 




13. 


How closely did instructor keep 
track? Very close Rarely 


e 


0.8 




14. 


Student- Instructor Relationship 




0.9 





Table 9: Results of Pre "and Post Test 
for Module V-a: {N=5) 





Mean 


Standard Deviation 


're Test 


23.2 


12.8 


Post Test 


74.0 


14.6 


Di fference 


50.8 


7.3 • 



t(4) =15.5 , P<.01 
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Analysis of Field Tests of All Modules 

A global assessment of the CCC project was attempted by summarizing 
information from the student opinion questionnaire across all 35 field 
tests. Table 10 presents descriptive statistics obtained by treating 
each field test as the unit of observation; that is, for Item one, the 
importance of the module objective, the mean of 3.53 is the average of 
all means for this item across all field tests. Relative to other items, 
the importance of the module objective was highly rated. This^implies that 
the students generally felt the objectives were relevant to their educa- 
tional goals. The means ranged from 2.3 for Module VII-c {Research Pro- 
cedures) to 4.8 for Module IV-b {Writing a Project/Budget). The objec- 
tive of Module V-c {instructional Strategies) was highly rated at one 
field site but not at the other two. The modules concerning Cooperative 
Relationships {ll-a,b,c) also received high ratings. 

Item two concerned the organization of the ^modules- The overall mean 
was 2.8 while the median was 2.6. This indicates that these data tend to 
have a positively skewed distribution; a few modules such as Il-b 
(Guidance and Counseling) and Il-d (Students with Special Needs) received 
high ratings, but most of the remaining modules received lower ratings. 

Item three, length of the module, received an overall mean of 3.52 
with a median of 3,40; again there is the indication of a positively skewed 
distribution. The students felt the modules were generally too long. 
Only two modules were rated towards the **too short*' end of the scale. 
ModuJe V-a (Curriculum Design) received a rating of 5.0 at two of tie 
three field test sites. 

The difficulty of the vocabulary. Item four, was generally rated as 
**about right/* The mean of the standard deviation (s) was 0.41; this was 
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the smallest value for this statistic across all Items and suggests that 
relatively little variability in vocabulary difficulty exists among the 
modules. 

Item five, availability of references, received the lowest mean rating 

T 

of all items (2.32); this was definitely towards the "not available" end 
of the scale. The mean of the standard deviations (s) was 0.99 and was 
the largest value for this statistic. There was much variability in the 
students' responses to ^this iteiti. Availability of references was a re- 
current theme heard both from students using the modules and from those 
Instructors who conscientiously tried to field test the modules. A perusal 
of many of the modules indicates that the authors provide a "bibliography** 
type reference page, but the lessons and activities are not intercalculated 
with these references. It would seem incumbent upon the authors to indi- 
cate specific resources for each lesson and each activity. In some cases 
where an author wrote more than one module, the same reference page is 
used in both modules. At best, this might indicate an extensive overlap 
in material covered by the modules; at worst, it may indicate that the 
author was simply lazy. Clearly, some major revision needs to be made in 
some modules to ensure that, before further field testing, the reference 
availability problem is alleviated. 

Item six, the number of activities used in the module, received a 
mean rating of 3.65 across all modules. Unfortunately, it is not clear 
whether the student rated this item based on the number of activities 
they actually completed, or on the total number of activities suggested 
by the module. It seems more likely that the latter was the basis of 
the rating. Means ranged from 2.9 for Module V-c (Instructional Strat- 
egies) to 4,7 for Module Ill-e (Disadvantaged Students). The students 



generally feit that there were too many activities; but the content 
validity raters, if anything, indicated that Enore activities were needed. . 
As originally conceived, students using the CCC Enodules were to complete 
aJi of the activities and the test was designed to assess the "successful 
completion of the activities as well as to assess mastery of the module 
objective. In only a few instances did the field tested students actually 
complete^!! the activities andi in ^ few cases, it is doubtful that any 
activities were completed. There is not enough information available 
from the field test instructors to analyze the relationship between 
number of completed activities and test performancei but future field 
tests might try to address this question. 

Item seveni test difficulty, received a mean rating of 3.59 indicating 
that students thought the tests were too difficult. Only the test? for 
Module Il-b (Assessing Trainee Characteristics) and IV-b (Writing a Project/ 
Budget) were rated in the "too easy" direction. In contrasti the content 
validity raters generally indicated the tests were "about right" in dif- " 
ficulty. Furtherrorei many of the field test instructors objected to 
the format of the tests (they wanted multiple choice items)i .the wording 
of the tests (ambiguous), the difficulty in scoring the tests (time- 
consuming and/or subjective) , and the answers provided by the author 
(wrong). In this evaluator*s opinioni the pre-post test is one of the 
major problems in the CCC approach. In theory, the test is to (1) be 
used for diagnostic placement such that a student can skip specific 
lessons or the entire module; (2) assess knowledge of the material 
covered in the module and (3) ultimately guarantee that the student has 
mastered the behavioral objective which forms the basis of the module. 
Such an instrument is extremely difftcuU to develop. In general, the 
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tests of the modules are achievement based such that a high score indicates 
knowledge about material covered in the module. Some authors link the test 
with the lessons so that the diagnostic aspect of the test might be attempted. 
However, none, of the field test instructors utilized this facet of the CCC, 
Perhaps such a truly sophisticated use of the modules lies in the future. 

Item eight asked the students to rate how well they felt they met the 
objective after completing the module. The mean response of 2.95 indicated 
that most stijdents thought they could meet it "adequately". Means ranged 
from 2.0 for V-a (Curriculum Design) atid 2A for III-c (Guidance and 
Counseling) to 4.0 for Module Hl-g (Student Organizations). At some 
field test sites, the pre-post statistical analysis indicated that little 
or no change in scores occurred; yet, students rated themselves as adequately 
meeting the objective. This item set a rather ambiguous task and, to many 
students, this type of self-assessment was undoubtedly difficult. 

Item nine queried the degree to which students liked the module approach; 
responses were pretty disappointing. The mean of 2.75 across all field 
tests does not seem to suggest that. students are wholeheartedly embracing 
this innovative approach to education. A subsequent analysis looks at 
potential contributing factors to the responses to this item. 

Item ten, the extent to which the Instructor monitored the students' 
progress, had a mean rating of 3.07; the median is 3,30. These means, thus, 
follow a negatively skewed distribution with some field test instructors 
receiving a very low rating. Indeed, two instructors received mean ratings 
of 1.0, but another received a mean of 4.7. Clearly, there was marked 
diversity among instructors. 

Finally, Item eleven concerning the student-instructor relationship, 
received the highest mean rating of all items (3.82). This item was 
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predicated on the assumption that the CCC modules Would be used in their | 
'Mndividualized" format with the instructor meeting with, and guiding, 
each student. Since this occurred rarely, if at all, this Item was 
probably Interpreted by the;,students as sort of an overall evaluation of 
the instructor and/or^the course* In the context of the field tests, the 
item is not very useful; but it does suggest that even though the students 
did not like the module approach, they did not genisralize these feelings 
to the instructor. 

Sunomary of Pre-Test'/Post-Test Changes 

With one exception* the data from the pre and post tests were analyzed 
using "two sample" t-tests for dependent groups. These tests indicate 
whether a "statistically significant'* change in the mean scores from pre to 
post test was observed. In 26 out of 35 field tests, a significant change 
at the .01 probability level was noted. Thus, in approximately 75% of the 
field tests, there is evidence that the students really did learn something. 
This is an impressive result and, although statistical quibbles and more 
serious design questions might be raised, the conclusion seems undeniab.le-- 
the Common Core Curri culum approach does work . 

In an effort to generate a more "sensitive" index of the outcome of 
the field tests than the dichotomoys decision provided by the statistical 
analysis, the evaluator developed his own five-point scale of "field test 
success". In obtaining this rating, the evaluator "subjectively** considered 
the magnitude of the t-test, thte magnitude of the change from pre to post 
test, and the initial starting pp1nt as indicated by the pre-test. A high 
rating indicates a large, significant increase from a relatively low 
starting point. A low rating indicates essentially no increase at all.. 




Table U indicates which field tests fell into each of the five rating 
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classes. Twelve field tests were rated as "highly successful"; in these 
cases* the students started with relatively low pre-test scores and dramat- 
ically increased their scores to a point where the behavioral objective, 
as measured by the test, would be met. Seven field tests were considered 
'^successful"; in most of these case^he students started with fairly high 
pre-test scores but still Increased, in these field tests* the class 
chosen to use the module may have been more advanced than the level for 
which the module was designed. In those cases* the students might have 
been able to skip some of the lessons. Six field tests were considered 
"moderately successful"; here a statistically significant increase was 
observed* but the actual gain was not very large or tho average post-test 
scores were. still well below the top scores obtainable. Two field tests 
were '^marginally successful"; in one case, no statistically significant 
increase was observed* but this was due to one or two studenvs actually 
decreasing scores from pre to post test. .Finally* eight field tests 
were rated as "not successful"; no statistically significant change was 
noted, ■ 

For tho5^ modules field tested at more than one site* inspection of 
the table indica . js that Module I-a {History* Philosophy) was highly 
f^uccessful at both places. Module Il-b (Advisory Councils) was highly 
successful at one site and successful at the other site. Module Ill-a 
{Promoting and Recruiting Students) was moderately successful at one site 
and only marginally successful at another site. Modules V-a,b,c were 
tested at three sites; ratings ranged from one to five for these modules, 
M6dule Vt-d {Safety) was rated well at both sites* but Module VlUb 
(Evaluating Local Programs) was unsuccessful at two field sites- 
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It is also possible to match the instructpr to the field test success 
ratings. The instructor who tried to use five modules had one moderately 
successful, one marginally successful, and three unsuccessful outcomes* 
Clearly, not enough time was devoted to each module. Another instructor - 
who tested four modules had two highly ^^uccessful and two unsuccessful 
outcomes- 
Intercorrelations 

Using the mean ratings for each item at the 35 field tests, an inter- 
correlation matrix, was generated and is presented in Table 12. A brief . 
description of the matrix follows. 

The rating of the importance of the objective (Variable 1) correlated 
highly with the organization of the module (V2) and the extent to which 
the students felt they met the objective (V8), Variable 1 also correlated 
with the students' liking of the module approach {V9). The organization of 
the module (V2) correlated highly with the availability of references and 
negatively with the number of activities; i.e., the more activities, the 
poorer the organization. Not very surprisingly, the length of the module 
(V3) correlated with the number of activities and the difficulty of the 
test (long tests were probably rated as more difficult). Vocabulary level 
(V4) was positively related to number of activities and difficulty of the ^ 
test. 

The availability of references {V5), as previously mentioned, was 
positively correlated with organization. The extent to which they met the 
objective, and the extent to which they liked the module approach, also . 
correlated with reference availability. Variable 6, number of activities, 
was negatively correlated with meeting the objective and with liking the 
approach. Test difficulty (V7), in addition to relationships with leng':h 
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and activities," was negatively correlated with ffeeting the objective. 

Variable 8» how well the objective was met, was related to organiza- 
tion, importance of the objective, and availability of references. The 
degree of enthusiasm for the module approach* V9, was positively correilated 
with organization* importance of objectiva, and availability of references. 
Variable 10, the extent to which the instructor monitored the activities 
of the students, showed generally weak relationships with all other vari- 
ables although it was positively correlated with liking the module approach 

Finally,, the field test success rating (VII), was poorly correlated^ 
with the opinion items. It had a marginal correlation with the length of 
the module* but that was about all- Apparently, whatever this success 
rating is measuring is not highly related to the module characteristics. 
Stepwise Multiple Regression 

One issue th^t must be considered is why the students liked (or did 
not like) the module approach in these field tests. In an attempt to 
discern some order among these, at times* chaotic data, a stepwise multiple 
regression was performed using Variable 9, the extent to which the module 
approach was liked, as the dependent variable* and the other items of the 
student questionnaire as independent variables. Table 13 provides some 
interesting results and generates a plausible model for indicating the 
contributing factors to student satisfaction with the module approach. 

The first five entering variables in the equation were: (l) organi- 
zation, (2) monitoring of progress, (3) meeting the objective, (4) number 
of activities and (5} length of module. Beyond this point* addition of 
further variables increased the squared multiple correlation less than 
1% and the F ratio for significance of the multiple correlation (R) was 
no longer significant. 
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Inspection of the table indicates that the two major contributing 
variables to Hking the module approach are organization of the module 
which accounted for 32% of the total variance, and the extent to which the 
instructor kept track of the students* progress, which added 6% to the 
total variance. The remaining^ three variables generated only small in- 
creases in the explained variance. 

It trfould seem most reasonable, to generalize from these data that 
there are two basic factors which ultimately influence the effectiveness 
and acceptance of. the module approach: 

(1) Characteristics of the module as typified by the module's 
organization" 

(2) Characteristics of the instructor as typified by monitoring 
progress. 

J' 

As was indicated in the reviews of the individual field tests, there 
were times when even the most highly motivated and enthusiastic instructor 
was frustrated by a poorly organized module with a badly designed pre. 
test and unavailable references. Likewise, even the best, most well- 
thought module fared badly in the hands of an indifferent instructor 
who distributed the module with vague instructions and non-existent 
guidance. 

The regression equation treats each of these factors as independent* 
but, in reality, they are not; there is a module-instructor interaction . 
Future use of the CCC modules must treat. each of these factors as equally 
important, and it may even be necessary to prepare a module that the-.in- 
structor completes which has as its behavioral objective the ability to 
use an individualized instructional method. * \ 
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A single group pre-test post-test design was used to assess the 
effectiveness of the newly developed Common Core Curriculum modules, A 
total of 35 ^'ield tests utilizing 24 of the 29 modules were conducted 
Instructors were given "free rejn" to use the modules In the manner they 
felt was most appropriate for their particular class. Some instructors 
carefully integrated the module into the course- while others simply gave 
the modules to the students with little or no guidance, 

A significant improvement in scores from pre to post test was^ noted 
in 26 of the 35 field tes^ts (75%),. Thus* there was strong evidence that 
students using the modules uid gain knowledge relevant to the behavioral 
objectives for which the modules were constructed. 

The reactions of the students to the modules varied widely but» in 
general* they felt the behavioral objectives were important. Many of 
the modules were rated as too long with too many activities and with 
tests that were too difficult. A serious recurring problem was noted: 
namely* the references uti-lized by the module were not available to the 
students. 

The students' reaction to the module approach was also varied. Sj5rhe 
students found it innovative and challenging while others thought it was 
frustrating and boring. The reasons for this variability seem to lie in 
an interaction between characteristics of the module (organization* length 
activities) and characteristics of the irvstructor (extent to which the 
instructor monitored the progress of the'student) . Some instructors 
appeared to confuse individualized instruction with the ^'self-contained 
or programmed learning" approach. It needs to be made clear in future \ 
use of these modules that they are not self-contained. 
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' ■ Table 10: Summary of Responses to Student Opinion Items for 35 Field 
Tests of 24 Different Hodi:les " 
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.83 


.30 


Oraanizati on 


2.80 


2.60 
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Table 11: Frequency Distribution of Ratings of Outcomes of Field Tests fay 
Module 



RATING 



' Highly 
Successful 



Successful 



Moderately 
■Successful 



Marginally 
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Ill-d 


V-a^ 




Ill-b 


Il-b 


. Ill-e 


V-b 




V-c 


Ul-g 


V~a 


VII-c 




Vl-a 


iV-b" 


V-c 


VU-e 


Vl-b 
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Table 12: Intercorrelatlon of Student Opinion Itons.and Success Rating for 35 
F ield Tests^ ^ 

VARIABLE 



Name 
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11- 


Success of 
Field Test 
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Table 13: Stepwise Multiple Regression for Predicting Extent to Which 
Modgle Approach was Liked " 
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.32 
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.63 
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.01 
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.65 


.42 
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V. 


.66 


■ .43 


.01 


.215 
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F;(5,29) =4,48, p<.01 
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SUMMARY ANO CONCLUSIONS 
Maurine Vander Griend 

Serving as listeners and observers were the CSUF department chairmen 
from Agricultural Industry and Education, Office Administration* and 
Industrial Arts and Technology. The first observation made was the major- 
ity of workshop participants found the group work stimulating and reward- 
ing. One observer said; 

"I observed that very positive and very good responses 
were being made by the field testing personnel present at tliis 
workshop. The feeling I got was that everyone came here to do 
business. The work sessions were "V€£if industrious and were/ 
most helpful especially to the module writers." 

The following comments were heard in the group work sessions. They 

are in no specific order and the group is not identified. Some of the 

[ 

comments were repeated by several groups and some were heard at every 
group visited. 

One concern dealt with resources--the availability of resources 
listed by the author. Many references were not readily available to stu- 
dents because they were not in the library and in some cases too many 
students had access to too few copies. One field-test instructor sug- 
gested that the use of references would be more effective if specific 
chapters or pages of a reference were given with each activity. Lead 
timp to work with librarian, 'or ordering materials and for duplication 
of materials would have been very beneficial and some of the outcomes of 
the field testing might have been different. 

When it came to the activities, the field instructors suggested that 
the activities be headed as '^suggested activities" or tf.at they be grouped 
as those that are essential and those that are suggested. The user should 
be permitted to adapt activities. Some of the field test instructors who 

O 
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had completed the review found some activities that were not based on the 
pre-test or related to the objective. 

Several concerns were also expressed about the pre-test for some of 
the modules. This included comments such as pre-test is too long— repre- 
sentative questions from each lesson sufficient; pre-test deals with too 
many facts rather than concepts; pre-test is difficult to score; some of 
the answers need to be expanded; and some questions need clarification. 
Many field-test instructors said they found the answer key most helpful. 

Helpful to the user of modules would be the clarification of the 
relationship between modules--is there a sequence or does each module 
stand alone or which ones stand alone or are a part of a series, Accom- 
panying this should be a clear statement of the intent and the intended 
level of instruction for each module. 

It was very*evident the modules were field tested under different 
situations such as for independent study* in a classroom setting for 
specified number of weeks or as additional assignment. Some field test 
instructors found that they did not have enough time or allow time for 
individual student/teacher interaction* especially as these modules are 
not self-contained. Others found that the modules provided for differen- 
tial instruction and allowed for a supplementary form of instruction 
enabling positive growth, 

The workshop for teacher educators from all vocational areas .as the 
vehicle for a good working relationship with many teacher educators from 
many institutions. Several teacher educators stated that they planned to 
adapt and ufe'much of the material reviewed. 

If there is an opportunity to revise and finalize this Common Core 
Curriculum* there will be available to all institutions usable curriculum 
material . 
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Guidelines for the use of these modules or detailed instruction to 
the users regarding the Intent of the module will also be prepared- As 
with any curricular material these materials will need to be continually 
revised and updated. This will become the responsibility of each user 
in * is/her own particular situation. 
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WORKSHOP 
ComoKXi Con Cunlculum of Vocational Edi*cattOfi 
Airport Hdldcy tnn — MoToUJ Aoom 
Ff#tf>o, CHIfofnls 

PURPOSE: 

To «kss«8S and advIM on 39 Common Cora Curricutum 
Vocational Education modular. 

To identify f^aaarch noed», dasign and procaas retatad lo 
vocational adiicstk>n. 

MooOav^Ularclin) 
9:00 Ragiatratlon — coffaa 
9:30 Introductiona — Gwan c. CooKe 
9:45 ThaAppllcahonofthaRaaaarchProceaalo 
ProtHefna Vocational Education 
J. Robert Warmbrod 
10:30 TakoaBreaK 

10:45 Ovacvfawof triaCommonCor^Cun^iculum 

WUiw Batn Kannath Moat^lor 

Ann Bau9f DwaynoSchramm 
L I oy d Do wiar Gay le Sot>o I )k 

Fran Hfrttlna Gary Wlnagar 

11.45 FialdtaatingofCCCModuiaa — Tarry N^woU 

t^,l5 Lunchaoo 

m5 Groop«ork(nvtawof spaclitc modulas) 

(g rou pa c hatred by writers) 
3:15 RaaavcnOeslon: Imptlcallons for htarTrratalion 

— Jh RobenWarmbroO 



Tuaadaf {March 22^ 

9:00 Soft^Rasearct) Needs in Vocational £dv>cation 
— J. Rot>eri Warmbrod 
Take a Break 

Group work (reviavM ot spec if modules) 
Luncheon 

S4J mmary of Group Work 
Paoal Frad Nury, FranN Schroeter. 

Bane Haggbladet Gwen Cooke 
Informal Questlonaand Anttwara 
J . Robert Warmbrod Panel membera 
Terry NewetJ Module writera 

Next stap with project — MaurlnaL Vandar Griend 
HAVE ASAFETRIPHOME.., 



9:45 
10.00 
12 i5 

1:00 



2:00 



3:00 
3 30 



Conaultant: 

J . Robert Warmbrod 

Professor, Agr^culturaJ Education ^ 

The Ohio Sta^a Untver^iW 

Module Wrttera: 

William Qaln 

Assiatant Director of Vocational Education r 
Fresno City Untlied School Dlstrlcl . Fresno 
Ann Bauer 

Associate Professor. Home Economics 
Calilornia State Unlvarsrtyp Fresno 
Uoyd Dowler 

Professor. Agricultural Education 
CallforniaStateUniversityr Fresno 
Frances Harklns 

Associate Prolessor, Home Economics 
Cahfornla State University, Fresno 
Kenneth Moshier 

AssiMant Professor, irhdusUiai Arts and Technoiogr 

California State University, Fresno 

O way ne Schram m 

Pro'e!isor, Office Administration 

California SI at a University, Fresno 

GayieSoboIlk 

ProfassoTi Office AdmtnislrallOn 
CaJtfornia Stale University. Fresno 

GaryWlnecar 

Associate Professor, Industrial Arts and Technology 
CallornlaStateUnlversityp f^resno ^ 
Pro|ecf Evaluator: 

Cal Wright Edi#catJon. Evaluation and Research Inc. 

Wofkahop Director: 
G wen Cooke 

Chairperson, l^pajrtrnent of Home Economics 
CailforniaStateUnlvarsily^ Fresno 

' AaaiatarvttoDlractor: 

Mauri ne Vender Griend 

Adjunct Professoi', Home Economics 

California Stale University p Frt^Sno ' 

Funded by: Vocational Education Inslruction Section 
CaTiforniaSlate Department of Education 
Part F * Education ProfassiOns 
Development Act 



FtaklT««lln»tivetor*: 

L. Jov4;« AmitCn S«]dl«b>ch Community Co N*0« 

Ann aiuvr ^ Catttomi« Stala Un'vwiliy , Fr««no 

Dr. BIN Wativy 9tomt\ — Catiromt* Stm OnWwilty. Chico 

S«ndraC/*Mw — C«t|forni« PDlyt*Chnic5tWaOnl;f«r«JtV. 

Di O. Gftn«0«vl« - UnlvwsltynfCalirQrn(*.LoB Ane«<« 
Or. HOwvO 0*ct(«f ^ S«n Jou Stata Unkwvlty 
Or OenniaA. p^rVii«n_S«nDk>0O5twaUnkwihv 
Mary M^ryv at E igtinw ^ Ptcinc Union CoilOQi 
Or LAwt*n»W Erlcksan — Unltf«fttlty of Catlfornia. Lo) AnffBlA* 
Oennla Hampton — CAlttofnli5taT«Onlv«r«liy. ChlCO 
Dr. Jtmaa Htrpw — JoM 5Ut« Unl^tilV 
Di.CtwiMJ in«ch«r^C«iirDmlaSiar«Onlv*rttty.Lo«4ngela) 
.Or, Irvin LalhrOP — CallfomliSUT« Unvlwi^ty. Long B«>Kti 
Or. JMnML*liing-.Umv«r&ltyot Caaiort\l«pOA<«k« 
. Di. John H.J^nn San Francitco Si«iaUn{var»ltr 
Ot.OlH0\ C. Lopai — S«n JOMStataOnivwiV 
O'^WUmar M*i«dkft— California Sttu Unlwiity. LoaAn^plaa 
Or Edmund M*nnion *^ CallfOfnIaStata unlwauy. Chico 
Of Jamaa A. M«rwi — CAil^>rnlaSt«taOftivw«lty. NonhrM^ 
Di Hal t^tratar* — 5«n 0\*qQ Stala ljnlv«n(ty 
Dr G.W.Maj(wa{l^SanJos«StaiaUnlvorBiiy 
JamAsB.Mayo^ Fr«nont'l^vntn«gionaiOccm>tuon»JProor*rTi 
Karan Ov^A -.^ P«pp«rdlfi«Unlv*r«lty 
Qr.Kannath Phi Mips — California St«ta Univ«r^tty. Lo« Angsm 
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Appendix B 

COMMON CORE CURRICULUM OF VOCATIONAL EDUCATION - 



1976 - 1977 



Introduction to Vocational Education 

History - Philosophy - Trends - Vocational Education 

Scope* Function and Organization 

Legislation 

Assessing the Oob Market and Employment Trends 

Cooperative Relationship ' 
Rationale for Cooperative Relationships 
Advisory Councils ^ 
Cooperative and Work Experience Programs 

Students 

Promoting Vocational Education and Recruiting Eligible 

Students for Vocational Education Programs 
Assessing Students' Personal ^^h;*^rqcteristics 
Guidance and Counseling 

Assisting Students with Special Needs in Vocational 

Education Programs 
Assessing the Needs of the Disadvantaged Student 
Developing Student Leadership Qualities in Vocational 

Education Programs 
Student Organizations 

Administration 

Fiscal Management of a Vocational Education Program 
Writing a Vocational Education Project/Budget 
Record Keeping in Vocational Programs 

Curriculum Design in Vocational Education 

Developing a Curriculum DesIgn^ in Vocational Education 
Learning Theory 
. Instructional Strategies 

Stages and Structure of Curriculum Development 
Cu rr 1 culum Theory in Vocational Education 
Buflding Curriculum in Vocational Education 
Curriculum Specifics in Vocational Education 
Safety 

Evaluation and Researc h . 
Evaluation ModeV 

Evaluation Procedures for Local Programs 

Introduction to Research Procedures in Vocational Educatioij 

Research Design in Vocational Education 

Development of a Research Proposal in Vocational Education 



CAUFQ^IA STATE UNI\'ERSITY • FI^NO 

FRESNO, CALIFORNJA 93740 
SCHOOL OF PR< )HtSS[ONAL STUDIES 



To: FIELD TEST INSTRUCTORS ^ 
COMMON CORE CURRICULUM FOR VOCATIONAL EDUCATION 

"rom: Or. Gwen Cooke, Chairperson 
Home Economics Department 
California State University, Frenno 
Fresno,. Cal ifornia 9374D 
(209) 487-2164 

Re: Instruc^ons for field testing modules 



1. In the packet 1s a copy of 

-each of the CCC Modules requested. (Modules may come in separate packets.) 
The instructor's copy includes pre-post tests, answer sheet and a total list of 
all suggested resources for the module. 

-module for students - The instructor will "^-^Pd to make a copy available for each 
student. This module does not include thw .le-post tests* answer sheet and a 
list of suggested resources for the module. 

-Content Validity Questionnaire. After reading the module carefully, but prior 
to actually using the module; the instructor will fill out the Content Validity 
Questionnaire: 

-Student Progress Check List to be completed by the instructor by filling in 
those items applicable to the particijlar module being tested. (Five copies 
are enclosed sufficient for a class of 30 members.) 

-"Ditto masters" of the Student Opinion and Satisfaction Questionnaire to be 
completed by each student participating in field test. 

-"Ditto masters'^ of the Student Background Data Sheet to be completed for each 
m#^ber by each student participating irt the field test. 

Th^ instructor will administer and score the pre-test. Be sure students put 
thelr^name* instructor's name and date in the proper bl^nk on the first page 
of the test. Have students write PRu-TEST at the top of t^e test. ^ 

When^ scoring the test, the instructor will record the number of points the stu- 
dent received for each questionnn front of the number of the question. The ' 
instructo r will total the score and put in front of the students* name on the 
first pagfe of the test. 

1. The instructor wi 1 1 moniten' the progress of the student througn the module. 
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CONTENT VALIDITY QUESTIONNAIRE 
COMMON CORE CURRICULUM MODULES 
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As part of 'our evaluation of the CCC project we are interested .in the content 
validity of the CCC modules. Content validity refers to the adequacy of the 
content of the module as rated by knowledgeable individuals in the area of , 
vocational education. After reading the module we would like you to complete^ 
this questionnaire' for each module.' Circle your response to each item. At 
the end of the questionnaire there is room for your written comments. 



MODULE TITLE' 



INSTRUCTOR'S NAME 
INSTITUTION 



NAME OF CLASS 



NUMBER OF STUDENTS ENROLLED, 
CLASS MEETINGS: 

HRS. PER WEEK 

TIME OF CLASS 



GRADUATE COURSE 



DAY(S) OF CLASS 



UNDERGRADUATE COURSE 



NUMBEg OF. WEEKS FOR THIS MODULE 



Part I: 

1, 



Behavioral Objective 

Indicate your evaluation of the behavioral objective which forms the basis 
of the module. . ' 



Clearly stated 
. 5 



Adequate 

3 



Vague 
■ 1 



2. How important is mastery of this objective for future vocational education 
teachers? 



Very 
5 



Somewhat 

3 V. 



Notv at a11 
1 



'art II: 

3, 



4 3 2 

The following questions concern the text and narrative sunwary.of the module: 
Organization of the module 



Excellent- 
.5 ^ 4 

4. Length of the module 

■* • 

Toe long 

5 ,4 

5. ' Vqcabulary level of the module 

Too difficult 
5 ■ 4 



6. References used In module 

4 



Readily available 
5 



Good 
3 



About right 
3 



About right 

3 



Available 

3 



Poor 
■ 1 



Too short 
.1 



Too simple 
1 



Not available 
1 
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7. Appropriateness of suggested use of references 

Excel lent f ' Adequate Poor 

5 4 3 '21 

8. Activities suggested by modules 

Too many About right Too few 

5 4 3 ^2 1 

.9. If you were to use this module approximately what percentage of the suggested 
activities do you think you would use? 

100% 755£ SOX 25% - 0% 

10. Range of activities 

Too broad About rlgh^t Too narrow 

5 1 4 3 2 ■ 1 ^ 

III: These questions concern the pre te^t posttest. 

11. Thp module pre-test is: 

Too difficult About right Too easy 

5 4 3 2 1 

12. With respect to the number of items in the pre tesf: 

Too many About right , Too few 

5 4 3 2 1 .^ 



13. With respect to scoring the test 



' Too difficult About right , Tot? easy 

5 ■ ' 4 \ 3 2 1 

IV: In general : \ 

14* If a student completes the module and passes the posttest** how well do you 
-feel the student would m^et trie behavioral objective? 

Excellent Adequate Marginally Could Not Meet 

■ 5 4 3 . 2 M 

15. Would you use the module in its present form? 

Definitely yes Probably yes Not sure Probably not Definitely not 
5 4. . 3 2 1 ^ 
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Conments : 



Text of Module: 



Pre/Post Test: 



Home Economtcs Department 
California State University, Fresno 
January 1977 
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STUDENT PROGRESS CH^CK LIST 
COMMON CORE CURRICilLUM MODULES 
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MODULE TITLE 



Activity 



Student 
.Code No. 



' Student 
Code No. 



Student 
Code No. 



Student 
Code No. 



Student 
. Code No. 



ri e test Ua ce 
-Score 












Lesson i s tdrtea 
□ate 
























L 












J 












4 






-i , 


~ 










\ — 1 












-A 1 






7 


r 




—4 






ES ^ 








i 








^- 1 








10 












• 11 
























Lessoh 1 completed 
• ^- ^date ^ 












lesson 2 started 
date 


4 






^ — 




Activity 1 












2 
























4 








1 fr- 




' 5 












6 ■ 












7 












8 
















1 








10 












ll 








4 


: ^ ^ 


■ . , 


. ' " . 










Lesson Z^completed 
date 
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STUDENT OPINION AND SATISFACTION QUESTIONNAIRE 
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COMMON CORE CURRICULUM MODULE 



Name of Module: 



After you have completed the module we would appreciate your responses to the 
following questions. Please circle the number which best indicates your response 

1. How important is mastery of the module objective for future vocational 
education workers? ^ 

Very important Somewhat Not at all 

5 ^4 3 2 1 \ 

2. What did you think of the organization of the module? 

Excellent ^ Good Poor 

5 4 3 2 1 - 

3. Length of module? 

Too long About right Too short 

5 4 3 2 1 ■ 

4. Vocabula'o' level of the module? 

Too difficult About right Too simple 

5 4 3 2.1 . . 

/ * ■ ■ 

5. References used inlnodule; 

Readily available Available Not available 



3 ^ ,2 1 



6., Activities suggested by the module. 

Too many About right Too few 

4 3" 2 1 ^ 

7. the module pre/post test was 

_ ^ Too difficult About right Tod easy 

5 4^3 2 "1 

8. Look at the behavioral objective wl^ich forms the basis of the module. 
How well do yoi* think you can behavlorally meet that objective? 

Very well ' ' Adequately . Not yery well 

O , . 5 , 4 3 2 1 , ■ 



9. How did you like the module-type approach to learning the material? 

Very much It was O.K. Did not ]i^e it 

5 4 .3 2 ' . 1 \ 

10. From which two activities do you feel you learned the most? 
1. 



2. 



11. From" which two activities do you feel you-learned the least? 



2. 



12. Can. you think of any further activities which might be used in this module? 
1. ' 



2. 



13. How closely -did your instructor keep track of your progress through the modul 

Constantly 

^Very closely Occasionally Rarely 

'5 4 , > 3 2 1 

14. How would you characterize the instructor-student relationship? ^ 

Excellent Good Poor • 

5 4 ^ 3\ 2 1 ' - 

is. Did you work on any of the activities with other students? 

Yes No If so* how c^id you like that? 

Enjoyed it ' / Would have rather done 

very much ^ / 0,K. activities glone 

5.4. 3 2 1 

Any other consents you might have about the module and/or the Comnon Core 
approach would be appreciated. Thank you very much for your help. 

\ Home Economics Department 

V : California State University* Fresno 

\ ' January, 1977 ^ 
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\ . STUDENT BACKGROUND DATA SHEET 



COMMON CORE CURRICULUM MODULES 



SEX UNDERGRADUATE STUDENT - 

ASE GRADUATE STUDENT , 

Ypur L^sua^^^^ad^ average (please circle) 

A A- ■ B B" C-^ C C- 

Program in which you are enrolled: 

1. Designated Subjected Credential in 



(subject area) 7 



2r**^achelor5 In : - Agriculture Business ^ Home Economics 

Industrial Education . Vocational Education 

3. Masters ^n: ^ Agriculture Business Home Economics 

'J 

Industrial Education Vocational Education 

if number three is checked please complete the following: * 

Agriculture^ 

Specialized Area:_ 

Home Economics 




Special ized Area: 



Industrial Education^ 
Specialized Area:_] 



Business Education^ 
^ Specialized Area1 



,9 
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4. Tfie I nstructor wil l administer and score the post-test 1n the same manfier as the 
Pre-'test. Have students write POST-TEST at the top oi;- the test. 

It 

5. The instructor vlll give the student the ''Student Opinior* and Satisfaction 
Questionna ire.*' The . students are not to write their namt? on, the in?itriiment. 
The questionnaire is anonymous. 

6' The ins tructor v^ill Send all Pre- ^nd Post Teb;ts, Content Validity Questionnaire, 
Student"OpTnTon Questionnaire, Progress Check-List and Baci;grouvid Datft Sheet to - 

Dr* Gwen Cooke, Chairperson 
Home Economics Department 

California State University, Fresno t 
Fresno, California 93740 

/. In some institutions, the same students will be co^opletlng m)re^ than one module. 
In thi$ case/the instructor will send complete information vor eachjnodule 
(i,e., each student will complete a. Student Background Data iheet for each iriodule). 

Any connents on the usefulness, problems encountered, etc.-, would be appreciated. 

GCC:cf . ■ ' - • . . ■ 



Home Economics Department 
California State University, Fresno 
January i?77 
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Appendix 0 
WORKSHOP PARTICIPANTS 

L. Joyce Arntson Saddleback Corwnunlty College 

wr.Harrs Bain - Fresno Unified School District, Fresno 

Ann Bauer - California State Uni/ersity, Fresno 

Jim Becxet - California State Department of Educationt Sacramento ^ 

Patti Bennett - California State Department of Ed-^cation, Sacramento 

Gwen Cooke - California State d^lversity, Fresno 

Howard Decker - San Jose State University, San Jose 

Lloyd Qowler - California State University, Fresno 

Mary Mai^garet Eighme - Pacific Union College. Angwin 

Gus Graham - California State University, Fresno 

Berle Haggblade - California State , University, Fresno - 

Frances Harkins - California State University^ Fresno ^ , 

Charles J. Inacker - California State Umversi^, bj% Angeles 

Irvin T* Lathrop - California State University, Long fieach 

James^i^sing - University of California^ Davis 

John Linn - Sarr Fraflcisco State Universit/, San Francisco 

Daniel C, Lopez - San Jose State University, San Jose 

>/ - ' ' . 

Wilmer Maedke - California State University, Los Angeles ^ 

I 

G. W. Maxwell - San Jose State University, San Jose 

Kenneth Moshier - California State^tJniversity, Fresno 

Terry, Newell - Califorpia State University,' Fresno 

Ratana Newsome -California State University, Frpsno ' ^ 

Troy Nuckols -San Francisco State University, San Francisco 

Fred Nury - California State University, Fresno 

Evelyn Price - California State University, Los Angeles 

Bonnie Rader- - California State University^, Long Beach 

Larry r. Rathbun - California Polytechnic ?tate University. San Luis Ob 

iOi) 
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Ed Rudloff -The Consortium of the California State University and Colleges, 
\^ Los Angeles , > , ^ 

Dwayne Schramm - CaUfornia State University* Fresno 

Frank Schroeter - California State Univer?ity> Fresno 

Qayle Sobolik - California State 'Jnlversity, Fresno" 

Garolyn Souza - California State University* Fresno 

Louise -Sutton - £crTiforn1a State Unlvcr^^ity* Northridge 

Leonard Torres - Cal Ifornia St^te University* Long Beach — \ 

Ruby Trow - California Polytechnic^^tate University* Pomona 

. Stan Urbanski - California State/ University* Long Beach 
Maurine Vander Griend - Cal Iforri^N^tate University* Fresno 
J, Robert Wannbrod - The Ohio State University 
James J* Weston - California State University* Sacramento 
Kathryn,lyhitten - Californi-a State Department of Ldy^^^onV Sacramento 
Marylynne Williams - Cal ifornia State Univers^ity * Lcs Angeles 

^ Gary Winegar - Cal ifornia. State University*rFtesno 
William Winnett - SanVranclsco State University* San Francisco 

- / 



.4^ 



f 



\ 
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FIELD TEST INSTRUCTORS 



L. Joyce Arnston - Saddleback Community College 

Sanclra Crabtree - CalTfornla Polytechnic State University, San Luis Dbispo 

Or, Howard Decker - San Jose State University 

Dr, Dennis A, Oirksen - San Diega State University 

Mary .Margaret Eighme - Pacific Union College 

Or, Lawrence W, Erickson - University of California, Los Angeles 

Dr. Charles J- Inacker - California State University, Los Angeles 

Dr.^rvin T, Lathrop - California State University, Long Beach 

Drv James Leising - University of California, Davis ■ 

Dr- John H, Linn - San Francisco State University ^ 

Or. Daniel C. Lopez^ San Jose State University 

Dr. Wilmer Haedke - California State University* Los Angeles 

Dr, James A. Manos - California State University, Northnidge 

Or, Hal Marsters - San Dijego State University , 

Dr. G. W, Maxwell - San Jo'se State University ^ 

Evelyn Trice - California State University, Los Angeles 

Bonnie Rader - California State University* Long Beach ^ 

Dr. Urry P. Rathbun - California Polytechnic State University, 

San Luis DbiSpo 

Louise .Sutton - California State University, Northridge 

Or. Leonard Torres - California State University , 4ong Beach 

Dr. Je<nes J, Weston - California State University, Sacramento 

Dr. Marylynne Williams - California. State University, Los Angeles 

Dr. William Winrtett - San Francisco State University , 

Dr. Kenneth Zimmer -talifornia State Univers.1ty, Los Angeles 



